Apparently Bill Belichick was a special teams coach under Parcells for 6 years before becoming a D-Coordinator.
I do think Special Teams coaches can do the job, but a guy like Rizzi who is mostly just a Special Teams coach for most of his career outside of a bit in college is going to have to have some really good coordinators and replacing them when they move on is always going to be an issue. It's not impossible, but it means you need to find two good to great coordinators every few years, not just one.
Dan Campbell for instance is going to have to show he can win without Johnson and Glenn at some point.