Saints owe New Orleans a season to say good-bye (an old headline)
I believe the problem is that people outside of the area have NO CLUE what this team means to us. I'm always hearing how the Packers and Steelers are such integral parts of their cities, but I'd put the connection between the Saints and New Orleans up against any team in the NFL. The Saints are part of our culture, not just a team. There is something between the fans and team that I'm not sure other cities and fans can understand.
I'm not going to say that losing the Saints back then would have killed the city, but it would have made it immensely more difficult to recover.