I agree - I wouldn't mind if this happens occasionally, but I don't want to deal with a variation of it every single time.
For me, I think what would be more interesting is the idea of civilization churn - old empires dying, and new ones being born - new races evolving and old races fracturing, potentially dying out entirely, tech loss and rediscovery over time.
This way, you do have the chance to become a Fallen Empire over time, but you have to work to keep it all from collapsing, because, you know, that's kind of what civilization does after a while - either politically or resource wise.