I wear my shorts and pants down around my behind or with a definite sag and I find it more comfortable than a pair of pants cinched around my waist.
All of my pants are at least a couple inches bigger than my natural waist size so that I can let them sag - I actually find them to be considerably more comfortable that way.
I could care less what sort of fashion statement it makes.