I agree. People may not know it but I have black friends and co-workers that get offended if you call them "African American" because they said they are not from Africa. The are "black". So how does one determine what is acceptable or offensive.
I know enough people who would rather be called Jamaican, Haitian, or are Aboriginal. So, I just use Black when it fits.
If someone wants to get upset with that and give me a hard time that I should call them an "African American" I'll just simply state that not every wants to be addressed that way, and it's not always appropriate to use. I don't know exactly where you came from.