Don't forget the Ravens were QB shopping as well....
The question is - would he have even been available?
The Chargers would have either signed him or tagged him. If they tagged him, they would have been required at least a number #1 to let him go.
He would have signed the offer sheet, btw. As dedicated as he is to the Saints - he was also that dedicated to the Chargers. And there wasn't a doubt in his mind that he would have beat out Rivers again in camp.