I think the plan laid out by the White House did outline guidance on when to implement each phase. Iirc, 2 weeks of sustained downward trend. I don't recall if that's based on number of deaths or number of cases though.
I think too soon would be when we have little room to respond to a spike in cases. You want to give yourself some flexibility to respond to an uptick or spike if opening back up turns bad.
I think it's tough to know what the metrics should be, but as a general rule, those are the thoughts I have.
And yes, I do think waiting a bit longer can do two things. One, ensure that the downward trend is real, and not a temporary dip, and two, further lower the number of overall cases so that if a spike in cases does happen, hospitals have a bit more capacity to work with when the spike happens.