Hello and welcome to Stellaris Performance Analysis #2. This issue is split in two (update: three) parts - one compares performance of different Stellaris versions; the other shows how some different galaxy setups impact performance of version 2.2.3. Update: third part shows impact of population and wars on performance. Same method is used as described in first issue. Unfortunately, save games will not be uploaded this time as uploading 22 GB will take some outrageous amount of time. However, specific saves can be available by request.
It should be specially highlighted once again that what is measured is, in fact, performance in observer mode. This doesn't translate one to one to actual gameplay performance but provides rather convenient way of obtaining data and is allows to measure impact of different game settings.
As well, a special remark about stuttering - data logger doesn't care if it took a minute (exaggerated) for a day to change with smooth 60 fps or with 5 fps.
Performance of different versions
After reading first issue, one might raise a rightful doubt about making conclusions based on single sample. To somewhat amend this, a set of 4 games is recorded and a mean value then is used for further comparison. While that is still quite small sample size, it should at least mitigate deviations due to activation of some hypothetical background services (e.g Windows Update, malware scanning, cat walking on keyboard).
Once again the performance is recorded in observer mode on the same settings: reset to default, set spiral 4 arms, set commodore difficulty, UNE empire. Versions covered: 2.1.2, 2.1.4, 2.2.2.1 and 2.2.3 (all plaza versions). Raw data and smoothened one are combined in single plot. Some plots have remarks.
A single game session from previous recordings.
A good example of crisis doing its job properly - due to total annihilation game was able to run smoothly all the way to year 2900.
Another example of total wipeout, running like a charm to year 3000. Crisis arrival caused some slowdown this time.
2.1.2 games showed more or less same performance all the way until crisis arrival in 2420s. Post crisis the plot behavior is somewhat similar, which is not a huge surprise given that all the crises were of the same type.
Second scourge appearance and this time with dramatic entrance - that's quite a spike even on smoothened plot.
For a change, spike occurred way past crisis arrival.
Again 2.1.4 shows nice performance similarity prior to crisis arrival. Post crisis behavior of scourges differs quite a lot between each other.
Here one can immediately see increase in spikes quantity and magnitude on non-smoothed plot.
This one actually doesn't look bad.
Typical scourge "mountain peaks" are present here and some dramatic jumps around years 2490 and 2620. These kind of jumps were observed when no sleep mode disabler was used during first data logging attempts. This might be caused either by game or by some background process suddenly feeling productive.
Smooth crisis arrival but a lot of spikes and once again a weird jump around year 2600.
In version 2.2.2.1 the deviation in performance between different runs occurs much earlier and is more severe - there might be more than 100ms difference between different games prior to crisis arrival. It looks like scourge crisis is more performance-heavy than unbidden.
Doesn't look like a lot of improvement compared to 2.2.2.1, but one should keep in mind remark about stuttering not being shown in these plots.
Clear winner is version 2.1.2 with 2.1.0 and 2.1.4 being a bit behind. Performance in 2.2.2.1 and 2.2.3 took quite a hit, the observer mode runs almost twice as slow in 2500 compared to previous versions.
While 2.2.3 seems to fixed early game performance, not much of improvement is seen past year 2400.
@Moah you were interested in version performance comparison, please find it above.
2.2.3 custom setups
Some custom galaxy setups were recorded to check what is the impact of certain game features. The setups are going to be briefly described first and their impact will be given next to comparison plot for better accessibility.
This setup has wormhole and gateway sliders set to 0.
This setup has marauder empires set to 0.
This setup has marauder and advanced empires set to 0. All the empires have Inwards Perfection civic by using forced spawn of custom-made empires.
First 200 years.
400 years in.
First custom setup was aimed at seeing the effect of wormholes and gateways as they were reported numerous times in performance issues threads. This brings performance to 2.1.4 levels for the first 100-150 years, but by year 2450 it almost catches up with regular games. The catch-up is most probably caused by FEs starting to build gateways, so it does indeed look like wormholes/gateways screw up performance.
Having overlooked the fact that FEs can build gateways (playing game is actually useful, should not only run test cases) second setup was made with assumption that migration might be an issue, and marauders were cut out of the galaxy to reduce slaves/refugees stream from raids/horde. This resulted in smoothest run in 2.2.3 so far.
Looking more into it, the first 150 years could be explained quite easily - in early game marauders are the sole users of wormholes. Most probably there were no gateways built (to be checked) by FEs either and the crisis was Unbidden - looking back at all the 2.2.2.1 and 2.2.3 data there is tendency for games with Unbidden to run faster. No surprises with that, as Scourge fails at properly wiping out planets.
But at that time the fact about FEs ability to build gateways/wormholes was still overlooked, so final setup was meant to be final nail in the migration coffin - all empires in the galaxy are Inwards Perfectionists, meaning no migration and refugees. Marauders were turned off as well, the future looked bright.
The result was the worst performance of all custom setups. Despite not having early wars, marauding raids (and thus as well early wormhole users) this galaxy matched regular setups in terms of performance. The most obvious explanation is the sheer amount of pops - being Inwards Perfectionist on top of Xenophobe cranks up pop growth ratio so it's a fair assumption.
Update: Link to part 3 (population and wars taken into account)
Results
Bottom line, the performance offenders so far identified in latest beta:
It should be specially highlighted once again that what is measured is, in fact, performance in observer mode. This doesn't translate one to one to actual gameplay performance but provides rather convenient way of obtaining data and is allows to measure impact of different game settings.
As well, a special remark about stuttering - data logger doesn't care if it took a minute (exaggerated) for a day to change with smooth 60 fps or with 5 fps.
Performance of different versions
After reading first issue, one might raise a rightful doubt about making conclusions based on single sample. To somewhat amend this, a set of 4 games is recorded and a mean value then is used for further comparison. While that is still quite small sample size, it should at least mitigate deviations due to activation of some hypothetical background services (e.g Windows Update, malware scanning, cat walking on keyboard).
Once again the performance is recorded in observer mode on the same settings: reset to default, set spiral 4 arms, set commodore difficulty, UNE empire. Versions covered: 2.1.2, 2.1.4, 2.2.2.1 and 2.2.3 (all plaza versions). Raw data and smoothened one are combined in single plot. Some plots have remarks.
A single game session from previous recordings.
2.1.2 games showed more or less same performance all the way until crisis arrival in 2420s. Post crisis the plot behavior is somewhat similar, which is not a huge surprise given that all the crises were of the same type.
Again 2.1.4 shows nice performance similarity prior to crisis arrival. Post crisis behavior of scourges differs quite a lot between each other.
In version 2.2.2.1 the deviation in performance between different runs occurs much earlier and is more severe - there might be more than 100ms difference between different games prior to crisis arrival. It looks like scourge crisis is more performance-heavy than unbidden.
Doesn't look like a lot of improvement compared to 2.2.2.1, but one should keep in mind remark about stuttering not being shown in these plots.
Clear winner is version 2.1.2 with 2.1.0 and 2.1.4 being a bit behind. Performance in 2.2.2.1 and 2.2.3 took quite a hit, the observer mode runs almost twice as slow in 2500 compared to previous versions.
While 2.2.3 seems to fixed early game performance, not much of improvement is seen past year 2400.
@Moah you were interested in version performance comparison, please find it above.
2.2.3 custom setups
Some custom galaxy setups were recorded to check what is the impact of certain game features. The setups are going to be briefly described first and their impact will be given next to comparison plot for better accessibility.
First 200 years.
400 years in.
First custom setup was aimed at seeing the effect of wormholes and gateways as they were reported numerous times in performance issues threads. This brings performance to 2.1.4 levels for the first 100-150 years, but by year 2450 it almost catches up with regular games. The catch-up is most probably caused by FEs starting to build gateways, so it does indeed look like wormholes/gateways screw up performance.
Having overlooked the fact that FEs can build gateways (playing game is actually useful, should not only run test cases) second setup was made with assumption that migration might be an issue, and marauders were cut out of the galaxy to reduce slaves/refugees stream from raids/horde. This resulted in smoothest run in 2.2.3 so far.
Looking more into it, the first 150 years could be explained quite easily - in early game marauders are the sole users of wormholes. Most probably there were no gateways built (to be checked) by FEs either and the crisis was Unbidden - looking back at all the 2.2.2.1 and 2.2.3 data there is tendency for games with Unbidden to run faster. No surprises with that, as Scourge fails at properly wiping out planets.
But at that time the fact about FEs ability to build gateways/wormholes was still overlooked, so final setup was meant to be final nail in the migration coffin - all empires in the galaxy are Inwards Perfectionists, meaning no migration and refugees. Marauders were turned off as well, the future looked bright.
The result was the worst performance of all custom setups. Despite not having early wars, marauding raids (and thus as well early wormhole users) this galaxy matched regular setups in terms of performance. The most obvious explanation is the sheer amount of pops - being Inwards Perfectionist on top of Xenophobe cranks up pop growth ratio so it's a fair assumption.
Update: Link to part 3 (population and wars taken into account)
Results
Bottom line, the performance offenders so far identified in latest beta:
- Wormholes and gateways (update: highly unlikely)
- Scourge (update: and contingency) failing at properly purging
- POPs related routines once enough have been grown
Attachments
Last edited: