I would like to somehow remove the white people from the equation, it would be fun to watch all the other groups fighting.

I suspect that if whites become a minority, they will start acting more like a minority, and that other minorities will proportionally focus their ire more on the other minorities that are in competition with them than on whites.

Anti-white discrimination needs to be countered, both legally and culturally however it should be done with a goal of having harmony between races rather then a fight for who can get the most benefits.

This is something that not just whites will agree with though.You certainly have Asians on your side on this matter, and judging by Proposition 18 in California, a majority of Latins and a significant minority of blacks.

I grew up not giving a shit about what color people were but I've came to the conclusion that a lot of people did even back then

They pretended not to give a damn, because it benefited their clients at the time, and now they have dropped the pretense, because that is now to their own political benefit.

Best example is how auditions for classical concerts were done blindly so that people could not discriminate against blacks, back in the day. Now professional grievancemongers are demanding that this practice be abolished so blacks can get privileges.