Sure. If statement A based on skin colour B is racist. Is the same exact same comment made by someone with skin colour B also racist when made about someone with skin colour C, D, E or F?
You'd argue that it isn't provided skin colour C, D, E or F were white and skin colour B is black. You're less certain if skin colour B is Asian and skin colour D, E, or F is black. Or if skin colour B continues to be black but C,D,E are brown. Personally, I reckon lumping racism and racial prejudice together and trying to eliminate both is better than this eternal game of guess the melanin level.
But it doesn't matter anyway, becuase as you've categorically stated, whiteness is available to all skin colours and the real pre-requisite to being 'white' is disagreeing with you.
I've heard of Ben Shapiro and I'm not a fan. Lefty remember. And I have no idea who Bari Weiss is.