At what point do you say something?

One additional thought...

If this is causing stress in your relationship, and you'd consider breaking up.. well, then you may want to have a chat before it gets that far. Or, just break up. Maybe this is the deal breaker.