I had a case recently where the circuit court, citing the need for the appearance of objective justice, replaced the district judge (after the second remand) on the basis that trial would require her to make credibility determinations of a witness she had already found credible in a plea hearing in a related criminal case.
Bottom line is they can do it if they want to. I think there's case law and ethics rulings that say that simply because a judge owns a reasonable (i.e. typical consumer) amount of stock in a public company, that isn't a financial conflict of interest. But it sure looks like one - and the appearance of conflict is supposed to be just as bad.