At what point do you say something?
One additional thought...
If this is causing stress in your relationship, and you'd consider breaking up.. well, then you may want to have a chat before it gets that far. Or, just break up. Maybe this is the deal breaker.