Performance since the patch has rendered the game unplayable

[ArmChairAttila]

OK updating, my game has made it to 2450 with some lag (more like stutter) but not the unplayable type I was getting in all my previous games. I have been running the BETA patches this whole time. I started using Glavius a few days ago but the late game lag still kicked in. It seems that either the bigger colonies + Glavius mods and/or max hyperlanes has significantly reduced the late game lag. It is definitely playable for me now.

Game settings I am using:
Small Galaxy
Max AI
2 FE
2 Marauders
Beta test patch
Bigger Colonies
Glavius bigger colonies patch
Grand Admiral
max hyperlanes
otherwise default

Links to mods below:
https://steamcommunity.com/sharedfiles/filedetails/?id=1584361127
https://steamcommunity.com/sharedfiles/filedetails/?id=1593318039

 

[totalkrieg]

Yes, game felt definitely slower after 100-150 years. Gateway AI pathfinding issue explains my problems perfectly. Before I knew I had switched to 600 star galaxies from 1000.

 

[Secret Master]

What's the "victory year" and map size that the game should be expected to run well on? Because I'm also wondering if Paradox didn't make a mistake on galaxy sizes. 1000-star galaxies with 300 year grand campaigns with 30 empires under the new 2.2.3 mechanics would be a tedious amount of micromanagement AND would likely bog down an NSA supercomputer and 3 cooperative players trying to play them.

But even a 400 star galaxy with a 0.75 crisis and a 2400 year victory got slow for me around 2415.

Even stranger, it kind of snuck up on me. Everything was fine, then the FE woke up and starting building gateways.

 

[Yandersen]

Looking at the thread I collected a few things that might be the reason for the end-game lag:

1) Pathfinding routine seem to inefficiently handling gateways. Probably a 1-way brute-force search algorithm is used, so a system with gateway is treated like it is connected to all other systems with gateways in terms of pathfinding. Effectively, more gateways means more branching for pathfinding algorithm. The right solution will be to find paths from both route ends to the nearest gateways and then search for the "land" route (ignoring gateways) to compare which way will be shorter. With proposed algorithm the pathfinding will take considerably less time lategame with multiple gateways. @Jamor

2) Variety of pops with different traits causing constant per-tick recalculations for better jobs shifting, migrations e.t.c. The more pops, the longer the recalculations will take.
I assume the per-tick system itself is a no-go solution. Must be event-based: pop recalculation on the planet must happen ONLY when certain condition changes to trigger it (number of pops changed, pops traits modified, building/district built/ruined, perk/tech/civic/policy/decision comes in effect, planet terraformed, colony changes it's type e.t.c.).

3) This one noone mentioned yet, but I have seen few times like fleets ordered to go another side of the galaxy sometimes stuck with destination hint looping between idle and certain gateway. Maybe there is some infinite loop bug causing some fleets to endlessly stuck eating up the performance? The latter the game, the more problematic fleets like that occur? Not sure what causes it, but it could be the change of border access while fleets are on route.

 

[serpentskirt]

I have run and recorded performance of quite a number of 2.2.3 (latest public beta) games over the holidays. While I'm still writing a nice summary I can comment on some of the points from post above.

Pathfinding routine seem to inefficiently handling gateways.

That doesn't seem to have a huge impact anymore. Performance difference between regular game and game without wormholes/gateways is negligible. Looks like I was mistaken, the difference is about 10%.

Variety of pops with different traits causing constant per-tick recalculations for better jobs shifting, migrations e.t.c. The more pops, the longer the recalculations will take.

I suspect that it is just pops in general - session with custom galaxy full of inward perfectionists (so mostly same races and no migrations - I thought the migration was a culprit behind slowdown) had the worst performance after 100 years.

 

[Yandersen]

That doesn't seem to have a huge impact anymore. Performance difference between regular game and game without wormholes/gateways is negligible.

Very interisting! How did you managed to get a no-gateway game? Is it some mod that removes the gateways? Can you clarify the testing conditions? Is it just a no-gateways start? Are you sure none were built few hundred years later? Surely the FE has tech for it. Am I right?..

If you are certain that lag is persistent with NO ANY gateways on a map lategame, then it is probably the pop calculations tick events. Most probably. Need more confirmations on that, if any1 else will try and confirm the same.

Still, lategame we also have a jump-drives. Hardly, but it maaaay also validate as a reason for the lagging of pathfinding algorithms. Much less probably, but still one to consider. Since PDX gave up to invest in QA on their side feeding us with raw betas to do that job ourselves, we have around a week before they return from their holidays. Let's be prepared to tell them about their bugs, right, guys?

Um, yeah, debugging. In Russia... Yeah.

 

[serpentskirt]

@Yandersen The method is the same as described here. No gateways/wormholes were indeed made by game setup setting.

Result plot from this game doesn't differ much from regular settings (3-5% faster past year 2400 at best, could be just noise due to wars/ect.) so I don't think pathfinding is a major contributor here. It seems to be more like 10%, which might make it worth looking into.

I must admit I have not yet looked if none were constructed, but up to year 2500 I haven't observed much deviations. It gets quite hard to look past that date because a) running one session to year 2600 takes about 16 hours and b) performance differs a lot depending on crisis (unbidden do good job at wiping population, unlike scourge) so it's hard to compare.

In general performance is the same as in version 2.1.4 up to year 2325, then it starts to get worse. In fact, version 2.1.4 performance stops getting worse past 2325, so I think by that time all planets have grown their pops there, while in 2.2.3 they still can grow further. The degradation linearly continues up to year ~2425, then crisis arrives and things get even worse. By that time it takes about half a second for day to pass on fastest speed in observer mode. If you get unbidden it gets to 550ms, in case of scourge can be as bad as 800ms. I have not seen any contingency in 2.2.x yet.

That gives me impression that pops are mainly slowing game down.

 

[ManelPatu]

The fact that gateways and subsequently trade do not appear to be the main causes of lag is disappointing since those could be given special attention if they were determined to be the main causes.
However to be fair the lag is nothing new, it's just gotten worse and worse over the years to the point where we are now. A pity, since that probably means it can't be fixed. Has anyone tried to fool around with the UI? I notice a significant difference in day to day ticks when I hide the outliner. Does anybody have the same results?

 

[stumason]

The fact that gateways and subsequently trade do not appear to be the main causes of lag is disappointing since those could be given special attention if they were determined to be the main causes.
However to be fair the lag is nothing new, it's just gotten worse and worse over the years to the point where we are now. A pity, since that probably means it can't be fixed. Has anyone tried to fool around with the UI? I notice a significant difference in day to day ticks when I hide the outliner. Does anybody have the same results?

There was a discussion about a year or so ago about some long time problem within the general engine that has manifested in all PDS games using Clauswitz that had something to do with the UI causing all sorts of performance issues and that it was an issue everyone kept away from as fixing it was a fluster cuck of a nightmare, until one PDS boffin may have found a fix in HOI4... It all went quiet after that, although no news if that could/would be transposed over to Stellaris or even if the issue presented in the game at all.

 

[stumason]

@Yandersen

That gives me impression that pops are mainly slowing game down.

Which is disappointing as one of the reasons for the planet change was to improve performance - but it seems the sheer number of POPs is now drowning the game. Before, you have only had a couple of hundred pops in your entire Empire, now you can have that many on one planet. I sincerely hope there is something that can be done, because as it stands, I'm never going to be able to play any game to the end game unless I have a whole week to kill on one game.

 

[Killeraoc]

The performance slogfest is just a cluster...makes the game impossible to play to completion which kills the enjoyment. How could it slip past QA? How does something like 2/3 of the end game crisis not being able to land on a planet, purge everything, and move on go live?

As empires grow in size and pop diversity you get 10+ empires with hundreds of pops constantly checking/reshuffling...and the problem scales with galactic pop growth/pop diversity.

That’s a major problem, on top of these AI pathfinding calculation hangs folks are mentioning here...which also scales in scope as AI gets more fleets/fleet power.

Without a fix for these issues it isn’t possible to finish a game without sitting through hours of choppy “lag”...things like brain dead AI and it not being able to make any use of the new economic system can at least be worked around. The lag? Nope.

 

[serpentskirt]

I need to withdraw my statement about wormholes/gateways. Looks like I was comparing it to the most smooth run of the standard settings session. When compared to mean over 4 sessions there is about 10% difference in favor of no wormholes/gateways setup.

I better stop ninjaing from the phone and finish my post.

 

[ragehavoc]

i get stuttering from the start, make a new non ironman game, type 3dstats into console, put game on fastest speed, and you will see the graph spike every tick, that is a huge engine issue.

 

[stumason]

The performance slogfest is just a cluster...makes the game impossible to play to completion which kills the enjoyment. How could it slip past QA? How does something like 2/3 of the end game crisis not being able to land on a planet, purge everything, and move on go live?

As empires grow in size and pop diversity you get 10+ empires with hundreds of pops constantly checking/reshuffling...and the problem scales with galactic pop growth/pop diversity.

That’s a major problem, on top of these AI pathfinding calculation hangs folks are mentioning here...which also scales in scope as AI gets more fleets/fleet power.

Without a fix for these issues it isn’t possible to finish a game without sitting through hours of choppy “lag”...things like brain dead AI and it not being able to make any use of the new economic system can at least be worked around. The lag? Nope.

I think part of the problem, at least, is the Devs appear to play against each other more often than not. I would like to think that's not the case and that someone does sit and play a SP game for 200 years, but if all you have is human players in a game, you haven't got half the issues you have trying to run 25 AI empires with all their shenanigans going on.

On top of that, in my most recent few games, the AI appears to have stopped trying - I was 50 years in and every single one of the AI empires, bar the FE's, still only had their 3 starting corvettes. They were all "pathetic" to me, even though I tried to gimp myself to allow them to catch up. They just refused to build fleets. I even switched to another AI and war dec'd another to try and prompt some kind of arms race or something- both just sat there, on full alloys no less and built nothing, moved nothing and killed nothing. Now that might be an issue with one of the 30 or so mods I am running, but I am not running many that diddle with the AI apart from Glavius, but no one else appears to have this issue..

 

[Yandersen]

Well, the stuttering due to overcrowding CAN be optimized. If pop shuffling logic moves from scripting into the actual code where modders will no longer be able to change anything. Since we are no longer able to manually shift pops anyway, it will not be such a big loss as before. The major concern about this approach is how buggy it will be if implemented in hard code - the modders will not be able to mod-patch it quickly as it always happen with PDX bug-updates... Oh, seems like I am getting into old cranky ways, sry.

 

[Jibril]

I don't care what mechanic caused this nor is it my job to find out. All I know is I bought DLC, game is stuttering. Stutters on both single player and multiplayer.
No mods & good PC.

Extremely disappointing, hopefully PDX gets their act together. Modders and speculation won't fix the underlying issues with the game.

 

[stumason]

Id be quite happy if PDS could just tell us they know what the issue is - it is more frustrating not knowing, than at least being aware of what is causing it. Apparently, they think it was an issue with trade routes, but even with the 2.2.3 beta, people are reporting lag still.

 

[stumason]

I don't care what mechanic caused this nor is it my job to find out. All I know is I bought DLC, game is stuttering. Stutters on both single player and multiplayer.
No mods & good PC.

Extremely disappointing, hopefully PDX gets their act together. Modders and speculation won't fix the underlying issues with the game.

Agreed. I am a PDS loyalist and Stellaris fanboy, but even my patience is wearing thin. CK2 can run like Usain Bolt on speed, even with thousands of characters and nations all doing their thing, but Stellaris lags a mere 1/4 of the way into a game and by the time you get to the part where you have developed empires and things get interesting, you're sitting there waiting for the days to tick by like it's on slow - why?????

 

[Killeraoc]

I think part of the problem, at least, is the Devs appear to play against each other more often than not. I would like to think that's not the case and that someone does sit and play a SP game for 200 years, but if all you have is human players in a game, you haven't got half the issues you have trying to run 25 AI empires with all their shenanigans going on.

On top of that, in my most recent few games, the AI appears to have stopped trying - I was 50 years in and every single one of the AI empires, bar the FE's, still only had their 3 starting corvettes. They were all "pathetic" to me, even though I tried to gimp myself to allow them to catch up. They just refused to build fleets. I even switched to another AI and war dec'd another to try and prompt some kind of arms race or something- both just sat there, on full alloys no less and built nothing, moved nothing and killed nothing. Now that might be an issue with one of the 30 or so mods I am running, but I am not running many that diddle with the AI apart from Glavius, but no one else appears to have this issue..

I suspect you're close to the truth here. And to make it worse I suspect that most of their "test games" never make it past the mid game...otherwise the performance on top of the broken AIs become so obvious that there's no way it could just "slip by"...just ignore it and ship it. Hell you don't even need to sit down and play a game to spot major functional problems here. Just use the Fast_Forward console command for a couple of games and even with no graphics you can observe the slowdown kick in midgame and scale with time. Jump into any simulated game and you'll see the AIs brokeness before un-pausing the game when you look at their economic setup. On top non-nonsensical and looped behavior. And you've got a 2/3 chance of observing a non-functional broken end-game crisis! That AI doesn't even have to worry about managing an economy! Its core behavior routine is spawn -> conquer -> invade -> purge -> move on to next system -> repeat. It can't even manage that.

 

[Yandersen]

Okay, here is an idea. For a new _.X._._ patch. To move away from individual discrete pops to a bulk population count, where discrete units represent not individual pops, but all pops of the same TYPE (same species, same traits = 1 pop type).

It means that there is no direct correspondence between a certain pop and a job slot: a production bonuses from a certain job type comes not directly from pops' traits, but from a statistics model:
if a planet's population of 1 billion pops, half of which is of type with +10% to mineral production and the other half has +10% to energy production, then both energy and mining jobs will produce the average base+5% resource production. Meaning all traits' bonuses get mixed into some "average unit" bonuses based on population' percentage of each individual species types presented on the planet. No pop shuffling. Just statistical average with little math, nothing else.

Each pop type has individual growth speed (+X heads per month, depending on factors like habitability, housing/amenities/jobs available and so on). On monthly tick each pop group grows/declines in number, after which the "average pop" model get it's bonuses recalculated based on new pop_types' percentage value.

As for the jobs, they should be filled up equally. Say, if one district provides 10 million jobs and we build 2 mining districts and 1 generator district, than we will have 20 millions mining jobs and 10 millions technician jobs (30 millions jobs available), but if the colony has only 15 million heads, then each of the district will be outputting at half capacity (with respective production bonuses depending on "average pop" model).

IMO such statistical model should work faster and be more realistic than the discrete pops we had since tiles' times. We wanted to get rid of tiles, but in fact, we just got them in different fashion in 2.2. Since the scale of the game we play is at galactic level, managing individual pops seems very illogical. Statistical models should replace the pops'n'tiles, IMO.