I think the difference between the racism that can be solved by awareness, and the racism that can't be is important.
I think people calling Joe Rogan racist are just trying to discredit him. There are plenty of other ways to do that though.
Calling people like that racist, only pushes them towards actual racists.