I was just looking at the text I sent him when he asked me for the 3rd date. I said "I've thought a lot about this over the past few days and you not being divorced makes this a situation I am unwilling to navigate together." He understood.
As a guy who was in that situation a long time ago, that was well stated; in such a way the guy doesn't feel like a loser but you also make your boundaries clear.