That's probably one reason why they always think they can get away with it. It shouldn't be tolerated just because it probably happens more than we think.
I really don't see anything wrong with wanting the guy to resign, given the fact that he preaches one thing and does the opposite. He's violating the trust of not only his family (which, I agree, is their problem), but also his constituency.
The kicker is that we're not talking about simple adultery. We're talking about solicitation of prostitution. However it is socially viewed, it's still illegal.