Hello everyone, today we would like to tease you with some of the upcoming changes coming with the 3.2 "Herbert" patch, named after Sci-Fi author Frank Herbert, which we will release along with the Aquatic Species Pack.
For Balance Changes, we have the following changes in store for you.
While not a pre-planned balance pass like we did for the Lem patch, we still found a few places to tweak and adjust and we will continue to do that in future patches.
...and now, handing over to Caligula Caesar for a look at some performance improvements and moddability topic.
A Look at Script Performance
Hi! You are probably used to me writing lengthy prose about new moddability and scripting language features. This time, we only have a few things to show off in that regard, but there are nevertheless some cool, technical things I can speak about.
Knowing the script language pretty well, I always found the performance impacts of our scripts to be a big unknown to me. Was what I was adding going to mess with performance? Well, I could do plenty of guessing as to how to script most efficiently, and general concepts of programming such as early outs do apply. But how big was the difference? And how much can we save by identifying inefficient scripts and improving them?
Moah had made some progress on porting the EU4 script profiler over to Stellaris as a pet project some time ago. The only problem was, its information was quite incomplete (since it needs a lot of tags added in many places of the code, basically everywhere where an effect or trigger is called). It was also pretty hard to read the information presented. But now, with the Custodians initiative, the time had come to see what we could do with this.
After a bit of (very tedious) work to make the information all-encompassing, systematic and readable, I let the game run on a Huge galaxy with a few extra boosts to the AI - 0.75 research costs, 1.25 habitable planets - and ran it a year with the script profiler enabled. Then, issues could be found. I’ve attached two versions of this output: one as it was in one of the early runs - so before coverage was comprehensive (notably, triggered modifiers and economic tables are missing), but also before any optimisation work was done - and one as it is now, in the 3.2 beta. (Note that the figures for how long it spent on each object is massively inflated by my having run the game in unoptimised debug mode with the profiler turned on)
Now, I must state in advance that we aren’t able to release the script profiler to the public with the 3.2 update for technical reasons: running the game with it makes the game about 50% slower, so we need to work out a way to be able to turn it - and its full performance impact - on and off at will. (At the moment, it is hidden behind compiler flags that are not available to the public). But we definitely hope that we’ll be able to release it to modders in the future.
Early Gains
The first big finding was that the game is repeatedly recalculating certain game rules a large number of times per pop each day, which was having a disproportionate impact on performance. The biggest culprit was “can_vote_in_democratic_election”, which it turned out was checked on every pop in the country every day for each pop faction while they were calculating their support value. Yes, you are reading this right: the imperialist faction would check whether each pop in the entire country was allowed to vote, then the prosperity faction would do the same, and the imperialist one, and so on… These cases were fixed by making use of daily caching: the pops will now calculate the result once per day (or, in the case of species_has_happiness, once per species in a country each day), and other places in the code can simply refer back to that result. Furthermore, pop factions’ support calculations were optimised so that the total by which they were dividing their support could be calculated once per country, rather than once per faction.
On the script side, by parsing various of the top hits, we noticed a few easily-optimised bits of script. First off, graygoo.555 was trying to fire a surprising number of times for an event that should come into play only when the Gray Goo are active (which they weren’t). It turns out that this was because it was missing “is_triggered_only”, so it was trying to fire on all planets every day! Similarly, a number of test events were scripted in a similar way, but with “always = no” as their trigger so they’d never fire. They made a small but nevertheless noticeable impact on performance, so they had to go.
The opinion modifier triggered_opinion_galactic_community_in_breach was taking up more performance than any other opinion modifier, by a distance, which seemed a bit strange. It turned out this could be fixed by a slight change in order in the triggers: it was checking “is_in_breach_of_any” before verifying Galcom membership - which sounds like it wouldn’t be a big issue, but that trigger then checks the triggers for the breach conditions of all passed resolutions, so it is in effect a lot of triggers in one. Simply swapping the order had very positive results, here.
Finally, the event crime.1 (somehow the second most costly event in the early version) was a similar case, but a lot more complicated. The main problem here was the following piece of script:
This is quite inefficient, and large benefits could be found in applying the principle of early outs. “Count_owned_pop” is a relatively expensive way of calculating anything, because a lot of efficiency is lost in converting script into code and working out the results of this, so on a planet with 80 pops, it is looping through each of those and checking a set of triggers on each of those. Unfortunately, because of the ordering, it would do this twice per day on each planet which did not have 3 unemployed pops on it:
Gains from Further Analysis
This was some cool stuff to fix, but beyond this, simply looking at the list became a bit harder to yield significant savings. Enter spreadsheeting! We pasted the results into a spreadsheet and, a few formulas later, and a nice pivot table to give us some breakdowns along the lines of “what is the total impact of all jobs”, or “what is the impact of the potential trigger of pop factions”
Picture shows values after performance improvements
This allowed us to pinpoint a few more things. Firstly, ai_resource_production was causing an absurdly high performance cost from a rather small number of hits. The culprit, here, turned out to be that the “planet_resource_compare” trigger (used mainly here) was incredibly expensive. The problem was that it was recalculating the resource output of all resources on the planet, basically (including by seeing what each pop was producing!). It turned out to be possible to mitigate this somewhat (to about 75%) by making it selectively recalculate the production of the relevant resource, but this was still quite expensive for a trigger, so we also cut down on its use a bit. I suggest modders not overuse it either.
Another thing we saw was that, not unexpectedly, jobs were quite expensive. Specifically their weights and their “possible” checks. We have some ideas to save time on the weights that we aren’t ready to speak about yet (they emerged too late in 3.2 development to be considered for the patch, because they are relatively likely to need some iteration), but we found a way of making the “possible” triggers cheaper. Basically, every seven days, a pop would recalculate its job cache, at which point it will check whether it is allowed to work each job, and if so, calculate its weight. But most jobs have fairly standard “possible” triggers that check the first part - specifically, there is a shared set of triggers between, respectively, worker, specialist, ruler and drone jobs. It turned out that very significant improvements (to the degree of almost two thirds) were possible by having the pop calculate these four triggers first, then loop through the jobs and simply match the result to the right job.
(Note to modders: the format looks a bit different now. If you used the scripted triggers worker/specialist/complex_specialist/ruler/drone_job_check_trigger, you will now need to define e.g. “possible_precalc = can_fill_ruler_job”. And if you changed them, you will need to change the new versions of them in game_rules)
Finally, although the game rules optimisations had already fixed several performance issues with pop factions, there were a few more spots where they could be improved. The first was whether a faction should exist at all: it turned out that both the script and the code was checking whether there were 5 pops that could join the faction, just that the code wasn’t checking this anymore after the faction was formed. Obviously, this wasn’t ideal, so the script check (being the slower) was removed, and the code check amended to account for shrinking pop factions becoming invalid.
The second was deciding whether a pop should belong to a faction: even though almost all factions only allow pops matching their ethos, the filter by ethos is quite late. By putting it much earlier - in code, before the script is even checked at all (with an override in case this isn’t desired, e.g. for technologist robots) - this massively cut down the costs of this particular calculation.
Finally, a number of their demands - checked each day per faction - were quite exorbitant. By changing the ordering and using equivalent but cheaper checks (e.g. any_owned_species instead of any_owned_pop). This, too, had a significant impact, so that the script footprint of pop factions (excluding game rules they use) was reduced by about two thirds.
Further Performance Topics
It is my hope that this work will be felt in the form of a bit less late game slowdown. My tests would indicate that this was a success, though it’s very hard to quantify by how much. It was however work that was solely focused on the script performance footprint, so there’s plenty of other things for us to look at! The job is never over, when it comes to performance, and hopefully we’ll have time to make further improvements for 3.3.
For example, I have heard a few complaints about UI lag in the late game, which might be improved slightly in a few interfaces as a result of the performance work, but this work didn’t focus on UIs. It is certainly true that some of them are not as fast as we would like them to be. Particular ones in this regard are the planet view, the species view and the colonisation selection menu, and we are looking at options to speed them up. (And, indeed, if anyone can think of any others, it would be useful for you to point them out!)
Moddability Improvements
I can’t really do a dev diary without talking about a few moddability improvements, so here they are. As I said, we don’t have that much this time, but there’s a few things that people might enjoy trying out:
There’s also a couple of things that modders will have to update (aside from the terraforming, as mentioned):
I am also, as last time, attaching the script docs to the dev diary, so that you can see any changes I forgot to mention. Also, any modders who are interested in early access to the 3.2 Update, for the purposes of getting your mods updated, you can sign up here: https://pdxint.at/3bZbVJN
For Balance Changes, we have the following changes in store for you.
- Functional Architecture and Constructobot: Reduced the free building slots granted from 2 to 1.
- Agrarian Idyll empires now get one planet building slot per four Agricultural districts built.
- Reduced the ship upkeep cost modifier for clone army admirals to 5/10/20% based on their decisions.
- Ruins of Shallash arc site no longer has a chance of giving quite as much unity as defeating an endgame crisis.
- You can no longer use planet killer weapons on primitives inside your borders if you lack the appropriate primitive interference policies.
- Pops working the Livestock job now have 10% less political power.
- A lot of anomalies were rewarding 3 Society Research deposits, there is now more variety.
- Made Awakened Fallen Empires use Traditions (but not Ascension Perks).
- Several productivity-improving technologies are now no longer of dubious benefit, as their upkeep (and production) effects now only apply to jobs actually primarily producing resources.
- Nerve Stapled Hivemind pops can no longer perform complex drone jobs.
- Reduced the amount of jobs added by Leisure Arcology Districts to bring them into line with other Ecumenopolis districts.
- Ion cannons are no longer free to maintain, and have an upkeep cost of 8 energy.
- Necro-Hives:
- Cut Necrophage pop assembly penalty to 50% from 75%
- Made pop output modifiers (positive and negative) no longer apply to hive minds.
- Made the -50% organic upkeep also apply to energy, for photosynthesis.
- Devouring Swarm Necrophages now spawn with extra infrastructure to account for the lack of chambers of elevation.
While not a pre-planned balance pass like we did for the Lem patch, we still found a few places to tweak and adjust and we will continue to do that in future patches.
...and now, handing over to Caligula Caesar for a look at some performance improvements and moddability topic.
A Look at Script Performance
Hi! You are probably used to me writing lengthy prose about new moddability and scripting language features. This time, we only have a few things to show off in that regard, but there are nevertheless some cool, technical things I can speak about.
Knowing the script language pretty well, I always found the performance impacts of our scripts to be a big unknown to me. Was what I was adding going to mess with performance? Well, I could do plenty of guessing as to how to script most efficiently, and general concepts of programming such as early outs do apply. But how big was the difference? And how much can we save by identifying inefficient scripts and improving them?
Moah had made some progress on porting the EU4 script profiler over to Stellaris as a pet project some time ago. The only problem was, its information was quite incomplete (since it needs a lot of tags added in many places of the code, basically everywhere where an effect or trigger is called). It was also pretty hard to read the information presented. But now, with the Custodians initiative, the time had come to see what we could do with this.
After a bit of (very tedious) work to make the information all-encompassing, systematic and readable, I let the game run on a Huge galaxy with a few extra boosts to the AI - 0.75 research costs, 1.25 habitable planets - and ran it a year with the script profiler enabled. Then, issues could be found. I’ve attached two versions of this output: one as it was in one of the early runs - so before coverage was comprehensive (notably, triggered modifiers and economic tables are missing), but also before any optimisation work was done - and one as it is now, in the 3.2 beta. (Note that the figures for how long it spent on each object is massively inflated by my having run the game in unoptimised debug mode with the profiler turned on)
Now, I must state in advance that we aren’t able to release the script profiler to the public with the 3.2 update for technical reasons: running the game with it makes the game about 50% slower, so we need to work out a way to be able to turn it - and its full performance impact - on and off at will. (At the moment, it is hidden behind compiler flags that are not available to the public). But we definitely hope that we’ll be able to release it to modders in the future.
Early Gains
The first big finding was that the game is repeatedly recalculating certain game rules a large number of times per pop each day, which was having a disproportionate impact on performance. The biggest culprit was “can_vote_in_democratic_election”, which it turned out was checked on every pop in the country every day for each pop faction while they were calculating their support value. Yes, you are reading this right: the imperialist faction would check whether each pop in the entire country was allowed to vote, then the prosperity faction would do the same, and the imperialist one, and so on… These cases were fixed by making use of daily caching: the pops will now calculate the result once per day (or, in the case of species_has_happiness, once per species in a country each day), and other places in the code can simply refer back to that result. Furthermore, pop factions’ support calculations were optimised so that the total by which they were dividing their support could be calculated once per country, rather than once per faction.
On the script side, by parsing various of the top hits, we noticed a few easily-optimised bits of script. First off, graygoo.555 was trying to fire a surprising number of times for an event that should come into play only when the Gray Goo are active (which they weren’t). It turns out that this was because it was missing “is_triggered_only”, so it was trying to fire on all planets every day! Similarly, a number of test events were scripted in a similar way, but with “always = no” as their trigger so they’d never fire. They made a small but nevertheless noticeable impact on performance, so they had to go.
The opinion modifier triggered_opinion_galactic_community_in_breach was taking up more performance than any other opinion modifier, by a distance, which seemed a bit strange. It turned out this could be fixed by a slight change in order in the triggers: it was checking “is_in_breach_of_any” before verifying Galcom membership - which sounds like it wouldn’t be a big issue, but that trigger then checks the triggers for the breach conditions of all passed resolutions, so it is in effect a lot of triggers in one. Simply swapping the order had very positive results, here.
Finally, the event crime.1 (somehow the second most costly event in the early version) was a similar case, but a lot more complicated. The main problem here was the following piece of script:
Code:
OR = {
AND = {
count_owned_pop = {
limit = {
is_shackled_robot = no
is_unemployed = yes
NOR = {
has_living_standard = { type = living_standard_utopian }
has_living_standard = { type = living_standard_good }
has_living_standard = { type = living_standard_shared_burden }
}
}
count > 3
}
owner = { is_gestalt = no }
}
AND = {
count_owned_pop = {
limit = {
is_unemployed = yes
NOT = { has_living_standard = { type = living_standard_organic_trophy } }
}
count > 10
}
owner = { is_gestalt = yes }
}
}- The event is checking the triggers every day on each inhabited planet. Or at least often. 44,000 times in a year, to be exact.
- It does not verify that there are unemployed pops on the planet before working out what kinds of unemployed pops there are. Which means that the OR will return false on both count_owned_pop sections, which consequently means it is checking both. Adding “num_unemployed > 3” near the start had big benefits
- It would check the number of unemployed pops relevant to non-gestalts and only after that check the country was not gestalt. By swapping the gestalt check to the start, it means it will only ever be trying one of the count_owned_pop loops.
Code:
num_unemployed > 3 #early out before the expensive count_owned_pop to come
OR = {
AND = {
owner = { is_gestalt = no }
count_owned_pop = {
limit = {
is_unemployed = yes
is_shackled_robot = no
NOR = {
has_living_standard = { type = living_standard_utopian }
has_living_standard = { type = living_standard_good }
has_living_standard = { type = living_standard_shared_burden }
}
}
count > 3
}
}
AND = {
owner = { is_gestalt = yes }
count_owned_pop = {
limit = {
is_unemployed = yes
NOT = { has_living_standard = { type = living_standard_organic_trophy } }
}
count > 10
}
}
}Gains from Further Analysis
This was some cool stuff to fix, but beyond this, simply looking at the list became a bit harder to yield significant savings. Enter spreadsheeting! We pasted the results into a spreadsheet and, a few formulas later, and a nice pivot table to give us some breakdowns along the lines of “what is the total impact of all jobs”, or “what is the impact of the potential trigger of pop factions”
Picture shows values after performance improvements
This allowed us to pinpoint a few more things. Firstly, ai_resource_production was causing an absurdly high performance cost from a rather small number of hits. The culprit, here, turned out to be that the “planet_resource_compare” trigger (used mainly here) was incredibly expensive. The problem was that it was recalculating the resource output of all resources on the planet, basically (including by seeing what each pop was producing!). It turned out to be possible to mitigate this somewhat (to about 75%) by making it selectively recalculate the production of the relevant resource, but this was still quite expensive for a trigger, so we also cut down on its use a bit. I suggest modders not overuse it either.
Another thing we saw was that, not unexpectedly, jobs were quite expensive. Specifically their weights and their “possible” checks. We have some ideas to save time on the weights that we aren’t ready to speak about yet (they emerged too late in 3.2 development to be considered for the patch, because they are relatively likely to need some iteration), but we found a way of making the “possible” triggers cheaper. Basically, every seven days, a pop would recalculate its job cache, at which point it will check whether it is allowed to work each job, and if so, calculate its weight. But most jobs have fairly standard “possible” triggers that check the first part - specifically, there is a shared set of triggers between, respectively, worker, specialist, ruler and drone jobs. It turned out that very significant improvements (to the degree of almost two thirds) were possible by having the pop calculate these four triggers first, then loop through the jobs and simply match the result to the right job.
(Note to modders: the format looks a bit different now. If you used the scripted triggers worker/specialist/complex_specialist/ruler/drone_job_check_trigger, you will now need to define e.g. “possible_precalc = can_fill_ruler_job”. And if you changed them, you will need to change the new versions of them in game_rules)
Finally, although the game rules optimisations had already fixed several performance issues with pop factions, there were a few more spots where they could be improved. The first was whether a faction should exist at all: it turned out that both the script and the code was checking whether there were 5 pops that could join the faction, just that the code wasn’t checking this anymore after the faction was formed. Obviously, this wasn’t ideal, so the script check (being the slower) was removed, and the code check amended to account for shrinking pop factions becoming invalid.
The second was deciding whether a pop should belong to a faction: even though almost all factions only allow pops matching their ethos, the filter by ethos is quite late. By putting it much earlier - in code, before the script is even checked at all (with an override in case this isn’t desired, e.g. for technologist robots) - this massively cut down the costs of this particular calculation.
Finally, a number of their demands - checked each day per faction - were quite exorbitant. By changing the ordering and using equivalent but cheaper checks (e.g. any_owned_species instead of any_owned_pop). This, too, had a significant impact, so that the script footprint of pop factions (excluding game rules they use) was reduced by about two thirds.
Further Performance Topics
It is my hope that this work will be felt in the form of a bit less late game slowdown. My tests would indicate that this was a success, though it’s very hard to quantify by how much. It was however work that was solely focused on the script performance footprint, so there’s plenty of other things for us to look at! The job is never over, when it comes to performance, and hopefully we’ll have time to make further improvements for 3.3.
For example, I have heard a few complaints about UI lag in the late game, which might be improved slightly in a few interfaces as a result of the performance work, but this work didn’t focus on UIs. It is certainly true that some of them are not as fast as we would like them to be. Particular ones in this regard are the planet view, the species view and the colonisation selection menu, and we are looking at options to speed them up. (And, indeed, if anyone can think of any others, it would be useful for you to point them out!)
Moddability Improvements
I can’t really do a dev diary without talking about a few moddability improvements, so here they are. As I said, we don’t have that much this time, but there’s a few things that people might enjoy trying out:
- There’s now a create_nebula effect. Although it’s best used during galaxy generation, since the visual effects on the galaxy map won’t refresh during the game.
- Decisions can now have on_queued and on_unqueued effects.
- Terraforming now uses the Megacorp economy system. Meaning, the costs are configurable resource tables, and you can make economic_unit modifiers to apply to them.
- There’s a species_gender trigger that checks what gender the species’ leaders can be
- You can now define custom tooltips for systems that you see when you mouse over them on the galactic map
- There’s now on_actions for on_capital_changed and on_planet_class_changed
- For Traditions, the “possible” trigger of its adopt tradition will now show in the tooltip for adopting it if it fails. I’m told that you can also now make the tradition categories have a container where you add a gridbox.
There’s also a couple of things that modders will have to update (aside from the terraforming, as mentioned):
- any/count/every/random_planet_army now refers to all armies on the planet, not just all owned by the owner of the planet
- Set_starbase_building/module and the remove variants are now consistent in starting at slot 1, rather than “set” starting at 0 and “remove” at 1.
- In component_templates, valid_for_country is now a trigger rather than a weights field
- Fire_on_action had some issues where you defined scopes with “prev” and “from”, those no longer exist.
I am also, as last time, attaching the script docs to the dev diary, so that you can see any changes I forgot to mention. Also, any modders who are interested in early access to the 3.2 Update, for the purposes of getting your mods updated, you can sign up here: https://pdxint.at/3bZbVJN
