He wasn't replaced BECAUSE of the color of his skin. Only racist think like that. He was cast because the showrunners thought he'd be great in the role.
The idea that the only reason this guy got the part is because he's black is inherently racist. They're not going to cast the person who plays Harry Potter with someone that's Jewish because the person's Jewish, they will do so because after auditions he was the best person according to the showrunners. No one is going to say he got the job because he's Jewish. Or substitute American, whatever. It's always skin color for racists.
Again, if someone's objection to the casting of a black person in a fantasy movie is, "it's because they're black", and not "they're not a good actor", it's racist. Only someone that's racist sees the former as a reasonable reason to object to a casting. Call it a racism cloud, but no being able to see WHY it's racist...is part of the problem.