There's not enough data on Allen as HC of the Saints to come to that determination. Youngest rookie head coach in the NFL for the most unstable franchise in the NFL version of Dennis Allen gets held against him far too often.
We get it, his overall record suggest he's a below average coach, no problem w/ that. Context makes that argument more fluffy however, so the best basis point for judging him is his 1st two years. That leaves him at middle of the roach coach after two years. A full evaluation of him will be more clear after the standard three years (no different with draft classes) and unless it's a total collapse it won't be enough to not give him a 4th year because he hasn't been atrocious.