If the client did it on his own, the only real risk is that it annoys the judge. Divorce litigation is by the bench (no juries), so the risk is that if the judge has to ever make a credibility determination (not unlikely in a divorce) or with any motion/request to the court where the goodwill of the client may be relevant - the judge isn't going to look favorably.
If the client's attorney advises the client to go interview the best lawyers in town to give them a conflict, you not only have the problem with the judge, I'd say that attorney has potentially violated the rules of professional conduct (probably rule 8.4).