Ooooh very interesting article and thanks for the tag. I don't actually have a problem with 'white people can't be racist' because as you've pointed out, it's part of a power imbalance that favours white people....
But... the greater power imbalance in a capitalist system is to be poor. I don't have a problem with people writing about race the way they do, I just don't think it's a good strategy for enacting change. Whilst your arguments are epistemically valid (did I use that word right?) I'm pretty sure they won't matter to people who don't have two brass half-pennies to rub together. The argument is valid, it just isn't useful.
Or, to quote you back to you, Don't assume just because it hurts to be unfairly insulted that you should complain loudly about not being allowed in the hang-gliding club at your Swiss retreat. Real people you're talking to have experienced bigger problems, and if you can't swallow this and move on then you can't expect them to do the same.
Buying into this distortion is like calling an ambulance for the paper cut. You make a lot of noise, waste a lot of time and forget about the people at the Earthquakes where the left-wing used to be.
I also (surprise surprise) don't agree that the right to define racism belongs to the people who experience it - because then it lacks definition and can be whatever we want it to be. You'll see trigger-creep across the issue and I'm pretty sure that's what happened, leading to Steve QJ writing a great article about racist hand-dryers. :o)