It's a bit counter intuitive, but the state of the current game balance also affects the Ai in a very negative way.
As far as I understand the metagame right now, if we look at the very early game, then the one strategy that doesn't involve rushing a neighbor but still leads towards a really strong game, is to find one of the extra planets and expand onto it asap. For a somewhat experienced player, that's not a problem because they can just hop through the systems, sees whether there are habitable planets in them and plan their expansion route according to the information they've gathered.
Now what does the AI do? It sends its Science ship into a random system, lets it survey that system independent from what's in there. If it's especially unlucky, it finds 1-2 anomalies that further delay the early exploration by a year or so. It only adds additional science ships at a delay, so the time that is wasted until the AI actually has their first planet can be immense. That's a serious problem for the AI, because these very first steps have such heavy snowballing potential. The AI's strengths and what's a good strategy are completely at odds with each other in this case.
Making the AI behave like a player isn't really an option I think, but altering the balance of those early gameplay options could very well lead to a more competitive AI, and have the bonus of giving players more options that don't feel like you're gimping yourself for the sake of roleplaying. There's really no need to have that rush towards an expansion be the one strong strategy that outperforms all alternatives so heavily.
This is true for other aspects of the game as well - traditions, ascension perks, technology choices, etc. - perfect balance will never exist of course, and the boost to the AI that can be gained from better balance is almost neglectable compared to the big problems its facing right now, but it would still help at no real cost.
I'll concede the point that the vanilla AI is even more trivial to beat if you play a min/max expansive/agressive game. And that yes, tuning the balance a bit in some places could remedy this and make for a landscape wheere a dynamic AI would be a more compleling expereince (/challange) to interact with.
... though, at its core, its a matter of doing the work on the AI scripting and pluggig holes to make sure it is possible for teh AI to use all the games features. On the former, i've come far enough working alone on an AI mod to know, its in reach.
in my expeirence its not a huge deal if an AI/any empire expands to a nearby planet in 2205 or 2208. unless there are very specific circumstances at play (like super crowded galaxy-settings) other things matter much more over the span of a game. Allowing the AI to recruit more scientists and build more science ships is a trivial task. (though, currently its irresponsible about leader / army upkeep - it has no means to be systematic in their recruitment or disbanding)
Obsering anecdote:
As an example, in a recent test-run i did with my mod, (observing the ai without plalying) one empire (corporation?) got cut down for size quite early and only had one planet. but because the difficuly was set to commodore (no scaling) and tech speed was fast, it got to habitats early and built several. it also joined / made a federation and ended up challanging its neighbour, taking space from them. Very cool to see.
Playing anecdote:
... i have only today began
playing stellaris again, after weeks modding - as i feel ok with where my AI-mod is at for now. so far im positively surprised.. im playing a militarist/materialist corporation. on scaling commodore with high research costs. I played an efficicently but not expensively early on, and pulled ahead on science score, managed to secure a vassal and some other safe local investments, but i struggled to become superior to my other neighbour whom i wanted to make a subsiduary... As i explored the medium galaxy more or less fully, by 2300, it became apperent that a few empires were ahead of me in score! their economies were significantly larger, but they were also keeping almost even in research... Im pleased with this, as it seems likely the galaxy will keep me interested for at least another 100 game years.
