Sorry, life isn't fair? Lol what kind of argument is that? You could say that about anything. Girls getting beat in sports by boys who identify as female, "Sorry, life isn't fair". Laws banning boys who identify as female from competing against natural born females, "Sorry, life isn't fair". The former would elicit little outrage but the latter would have people up in arms.
It's true, life isn't fair but it isn't fair to anyone. You can say, "Sorry, life isn't fair" but if someone said the same thing about women dealing with advances from men following their most basic biological imperative, it would be inappropriate and offensive.