My thoughts:
Pop growth should really care about just a few factors:
- Amenities
- Available Housing
- Available Jobs
- Planetary Capacity
- Habitability
If there is an excess of these on a planet then pop growth will occur at a faster rate. If not, then it slows down until eventually hitting 0 (stable population). When available, pops will also take into account other planets they are able to migrate to, though this will always be at a growth penalty relative to local growth factors.
This would have the interesting side-effect where you could pre-buy districts/buildings on a planet to temporary prop up it's pop growth; lots of jobs/housing would lead to a faster growth rate, allowing planets colonized later to catch up in population over a shorter timespan.
This "works" in the fact that, disregarding Habitats (which are their own issue) that overall pop capacity will only be slightly larger (10-20% is probably a good target) the total planetary capacity of the galaxy and serves as a hard cap on pop growth.
Habitats also need to be significantly scaled back; the AI spams them, and they lead to unchecked pop growth that simply breaks things. The easy fix is to limit Habitats to either Void Dweller origin or the VoidBorne ascension perk. Even then, I'd jack up their upkeep (especially for empires who go the ascension route) and add some other downsides (maybe pops born on Habitats having a negative modifier to Habitability, making it harder to move them around?)