Is the number of countries boycotting the Anti-Racist Conference racist in itself and nullifies the process?

Why is Brown attending - his administration has made it almost illegal to even speak to non-Brits already.