• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #232 - 3.2 Balance Changes and Performance Improvements

Hello everyone, today we would like to tease you with some of the upcoming changes coming with the 3.2 "Herbert" patch, named after Sci-Fi author Frank Herbert, which we will release along with the Aquatic Species Pack.

For Balance Changes, we have the following changes in store for you.
  • Functional Architecture and Constructobot: Reduced the free building slots granted from 2 to 1.
  • Agrarian Idyll empires now get one planet building slot per four Agricultural districts built.
  • Reduced the ship upkeep cost modifier for clone army admirals to 5/10/20% based on their decisions.
  • Ruins of Shallash arc site no longer has a chance of giving quite as much unity as defeating an endgame crisis.
  • You can no longer use planet killer weapons on primitives inside your borders if you lack the appropriate primitive interference policies.
  • Pops working the Livestock job now have 10% less political power.
  • A lot of anomalies were rewarding 3 Society Research deposits, there is now more variety.
  • Made Awakened Fallen Empires use Traditions (but not Ascension Perks).
  • Several productivity-improving technologies are now no longer of dubious benefit, as their upkeep (and production) effects now only apply to jobs actually primarily producing resources.
  • Nerve Stapled Hivemind pops can no longer perform complex drone jobs.
  • Reduced the amount of jobs added by Leisure Arcology Districts to bring them into line with other Ecumenopolis districts.
  • Ion cannons are no longer free to maintain, and have an upkeep cost of 8 energy.
  • Necro-Hives:
    • Cut Necrophage pop assembly penalty to 50% from 75%
    • Made pop output modifiers (positive and negative) no longer apply to hive minds.
    • Made the -50% organic upkeep also apply to energy, for photosynthesis.
    • Devouring Swarm Necrophages now spawn with extra infrastructure to account for the lack of chambers of elevation.
Of note here is the Functional Architecture change, we are aware that the extra building slot was the main draw of the civic but it was also way over-represented even after the initial release-hype

While not a pre-planned balance pass like we did for the Lem patch, we still found a few places to tweak and adjust and we will continue to do that in future patches.

...and now, handing over to Caligula Caesar for a look at some performance improvements and moddability topic.

A Look at Script Performance
Hi! You are probably used to me writing lengthy prose about new moddability and scripting language features. This time, we only have a few things to show off in that regard, but there are nevertheless some cool, technical things I can speak about.

Knowing the script language pretty well, I always found the performance impacts of our scripts to be a big unknown to me. Was what I was adding going to mess with performance? Well, I could do plenty of guessing as to how to script most efficiently, and general concepts of programming such as early outs do apply. But how big was the difference? And how much can we save by identifying inefficient scripts and improving them?

Moah had made some progress on porting the EU4 script profiler over to Stellaris as a pet project some time ago. The only problem was, its information was quite incomplete (since it needs a lot of tags added in many places of the code, basically everywhere where an effect or trigger is called). It was also pretty hard to read the information presented. But now, with the Custodians initiative, the time had come to see what we could do with this.

After a bit of (very tedious) work to make the information all-encompassing, systematic and readable, I let the game run on a Huge galaxy with a few extra boosts to the AI - 0.75 research costs, 1.25 habitable planets - and ran it a year with the script profiler enabled. Then, issues could be found. I’ve attached two versions of this output: one as it was in one of the early runs - so before coverage was comprehensive (notably, triggered modifiers and economic tables are missing), but also before any optimisation work was done - and one as it is now, in the 3.2 beta. (Note that the figures for how long it spent on each object is massively inflated by my having run the game in unoptimised debug mode with the profiler turned on)

Now, I must state in advance that we aren’t able to release the script profiler to the public with the 3.2 update for technical reasons: running the game with it makes the game about 50% slower, so we need to work out a way to be able to turn it - and its full performance impact - on and off at will. (At the moment, it is hidden behind compiler flags that are not available to the public). But we definitely hope that we’ll be able to release it to modders in the future.

Early Gains
The first big finding was that the game is repeatedly recalculating certain game rules a large number of times per pop each day, which was having a disproportionate impact on performance. The biggest culprit was “can_vote_in_democratic_election”, which it turned out was checked on every pop in the country every day for each pop faction while they were calculating their support value. Yes, you are reading this right: the imperialist faction would check whether each pop in the entire country was allowed to vote, then the prosperity faction would do the same, and the imperialist one, and so on… These cases were fixed by making use of daily caching: the pops will now calculate the result once per day (or, in the case of species_has_happiness, once per species in a country each day), and other places in the code can simply refer back to that result. Furthermore, pop factions’ support calculations were optimised so that the total by which they were dividing their support could be calculated once per country, rather than once per faction.

On the script side, by parsing various of the top hits, we noticed a few easily-optimised bits of script. First off, graygoo.555 was trying to fire a surprising number of times for an event that should come into play only when the Gray Goo are active (which they weren’t). It turns out that this was because it was missing “is_triggered_only”, so it was trying to fire on all planets every day! Similarly, a number of test events were scripted in a similar way, but with “always = no” as their trigger so they’d never fire. They made a small but nevertheless noticeable impact on performance, so they had to go.

The opinion modifier triggered_opinion_galactic_community_in_breach was taking up more performance than any other opinion modifier, by a distance, which seemed a bit strange. It turned out this could be fixed by a slight change in order in the triggers: it was checking “is_in_breach_of_any” before verifying Galcom membership - which sounds like it wouldn’t be a big issue, but that trigger then checks the triggers for the breach conditions of all passed resolutions, so it is in effect a lot of triggers in one. Simply swapping the order had very positive results, here.

Finally, the event crime.1 (somehow the second most costly event in the early version) was a similar case, but a lot more complicated. The main problem here was the following piece of script:

Code:
OR = {
            AND = {
                count_owned_pop = {
                    limit = {
                        is_shackled_robot = no
                        is_unemployed = yes
                        NOR = {
                            has_living_standard = { type = living_standard_utopian }
                            has_living_standard = { type = living_standard_good }
                            has_living_standard = { type = living_standard_shared_burden }
                        }
                    }
                    count > 3
                }
                owner = { is_gestalt = no }
            }
            AND = {
                count_owned_pop = {
                    limit = {
                        is_unemployed = yes
                        NOT = { has_living_standard = { type = living_standard_organic_trophy } }
                    }
                    count > 10
                }
                owner = { is_gestalt = yes }
            }
        }
This is quite inefficient, and large benefits could be found in applying the principle of early outs. “Count_owned_pop” is a relatively expensive way of calculating anything, because a lot of efficiency is lost in converting script into code and working out the results of this, so on a planet with 80 pops, it is looping through each of those and checking a set of triggers on each of those. Unfortunately, because of the ordering, it would do this twice per day on each planet which did not have 3 unemployed pops on it:
  • The event is checking the triggers every day on each inhabited planet. Or at least often. 44,000 times in a year, to be exact.
  • It does not verify that there are unemployed pops on the planet before working out what kinds of unemployed pops there are. Which means that the OR will return false on both count_owned_pop sections, which consequently means it is checking both. Adding “num_unemployed > 3” near the start had big benefits
  • It would check the number of unemployed pops relevant to non-gestalts and only after that check the country was not gestalt. By swapping the gestalt check to the start, it means it will only ever be trying one of the count_owned_pop loops.
A new, more efficient version of the trigger was therefore this:

Code:
num_unemployed > 3 #early out before the expensive count_owned_pop to come
        OR = {
            AND = {
                owner = { is_gestalt = no }
                count_owned_pop = {
                    limit = {
                        is_unemployed = yes
                        is_shackled_robot = no
                        NOR = {
                            has_living_standard = { type = living_standard_utopian }
                            has_living_standard = { type = living_standard_good }
                            has_living_standard = { type = living_standard_shared_burden }
 
                        }
                    }
                    count > 3
                }
            }
            AND = {
                owner = { is_gestalt = yes }
                count_owned_pop = {
                    limit = {
                        is_unemployed = yes
                        NOT = { has_living_standard = { type = living_standard_organic_trophy } }
                    }
                    count > 10
                }
            }
        }

Gains from Further Analysis
This was some cool stuff to fix, but beyond this, simply looking at the list became a bit harder to yield significant savings. Enter spreadsheeting! We pasted the results into a spreadsheet and, a few formulas later, and a nice pivot table to give us some breakdowns along the lines of “what is the total impact of all jobs”, or “what is the impact of the potential trigger of pop factions”

script profiler 1.png

Picture shows values after performance improvements

This allowed us to pinpoint a few more things. Firstly, ai_resource_production was causing an absurdly high performance cost from a rather small number of hits. The culprit, here, turned out to be that the “planet_resource_compare” trigger (used mainly here) was incredibly expensive. The problem was that it was recalculating the resource output of all resources on the planet, basically (including by seeing what each pop was producing!). It turned out to be possible to mitigate this somewhat (to about 75%) by making it selectively recalculate the production of the relevant resource, but this was still quite expensive for a trigger, so we also cut down on its use a bit. I suggest modders not overuse it either.

Another thing we saw was that, not unexpectedly, jobs were quite expensive. Specifically their weights and their “possible” checks. We have some ideas to save time on the weights that we aren’t ready to speak about yet (they emerged too late in 3.2 development to be considered for the patch, because they are relatively likely to need some iteration), but we found a way of making the “possible” triggers cheaper. Basically, every seven days, a pop would recalculate its job cache, at which point it will check whether it is allowed to work each job, and if so, calculate its weight. But most jobs have fairly standard “possible” triggers that check the first part - specifically, there is a shared set of triggers between, respectively, worker, specialist, ruler and drone jobs. It turned out that very significant improvements (to the degree of almost two thirds) were possible by having the pop calculate these four triggers first, then loop through the jobs and simply match the result to the right job.

(Note to modders: the format looks a bit different now. If you used the scripted triggers worker/specialist/complex_specialist/ruler/drone_job_check_trigger, you will now need to define e.g. “possible_precalc = can_fill_ruler_job”. And if you changed them, you will need to change the new versions of them in game_rules)

Finally, although the game rules optimisations had already fixed several performance issues with pop factions, there were a few more spots where they could be improved. The first was whether a faction should exist at all: it turned out that both the script and the code was checking whether there were 5 pops that could join the faction, just that the code wasn’t checking this anymore after the faction was formed. Obviously, this wasn’t ideal, so the script check (being the slower) was removed, and the code check amended to account for shrinking pop factions becoming invalid.

The second was deciding whether a pop should belong to a faction: even though almost all factions only allow pops matching their ethos, the filter by ethos is quite late. By putting it much earlier - in code, before the script is even checked at all (with an override in case this isn’t desired, e.g. for technologist robots) - this massively cut down the costs of this particular calculation.

Finally, a number of their demands - checked each day per faction - were quite exorbitant. By changing the ordering and using equivalent but cheaper checks (e.g. any_owned_species instead of any_owned_pop). This, too, had a significant impact, so that the script footprint of pop factions (excluding game rules they use) was reduced by about two thirds.

Further Performance Topics
It is my hope that this work will be felt in the form of a bit less late game slowdown. My tests would indicate that this was a success, though it’s very hard to quantify by how much. It was however work that was solely focused on the script performance footprint, so there’s plenty of other things for us to look at! The job is never over, when it comes to performance, and hopefully we’ll have time to make further improvements for 3.3.

For example, I have heard a few complaints about UI lag in the late game, which might be improved slightly in a few interfaces as a result of the performance work, but this work didn’t focus on UIs. It is certainly true that some of them are not as fast as we would like them to be. Particular ones in this regard are the planet view, the species view and the colonisation selection menu, and we are looking at options to speed them up. (And, indeed, if anyone can think of any others, it would be useful for you to point them out!)

Moddability Improvements
I can’t really do a dev diary without talking about a few moddability improvements, so here they are. As I said, we don’t have that much this time, but there’s a few things that people might enjoy trying out:
  • There’s now a create_nebula effect. Although it’s best used during galaxy generation, since the visual effects on the galaxy map won’t refresh during the game.
  • Decisions can now have on_queued and on_unqueued effects.
  • Terraforming now uses the Megacorp economy system. Meaning, the costs are configurable resource tables, and you can make economic_unit modifiers to apply to them.
  • There’s a species_gender trigger that checks what gender the species’ leaders can be
  • You can now define custom tooltips for systems that you see when you mouse over them on the galactic map
  • There’s now on_actions for on_capital_changed and on_planet_class_changed
  • For Traditions, the “possible” trigger of its adopt tradition will now show in the tooltip for adopting it if it fails. I’m told that you can also now make the tradition categories have a container where you add a gridbox.

There’s also a couple of things that modders will have to update (aside from the terraforming, as mentioned):
  • any/count/every/random_planet_army now refers to all armies on the planet, not just all owned by the owner of the planet
  • Set_starbase_building/module and the remove variants are now consistent in starting at slot 1, rather than “set” starting at 0 and “remove” at 1.
  • In component_templates, valid_for_country is now a trigger rather than a weights field
  • Fire_on_action had some issues where you defined scopes with “prev” and “from”, those no longer exist.
As a final moddability note, for anyone who misses the meaty dev diaries with far-reaching moddability changes, not to worry! Anyone that has played around with the script of our newer games will know that there’s a lot more potential in our scripting language. There’s some cool stuff in the works, though I can’t at this stage say what exactly or in which patch it’ll be.

I am also, as last time, attaching the script docs to the dev diary, so that you can see any changes I forgot to mention. Also, any modders who are interested in early access to the 3.2 Update, for the purposes of getting your mods updated, you can sign up here: https://pdxint.at/3bZbVJN
 

Attachments

  • profiling_summary after.log
    695,8 KB · Views: 0
  • profiling_summary before.log
    465,4 KB · Views: 0
  • effects.log
    181 KB · Views: 0
  • localizations.log
    4,4 KB · Views: 0
  • modifiers.log
    332,5 KB · Views: 0
  • scopes.log
    8,1 KB · Views: 0
  • triggers.log
    128,8 KB · Views: 0
Last edited by a moderator:
  • 95Like
  • 25Love
  • 25
  • 4
  • 2
Reactions:
My consistent experience of sabotage (rather than just espionage) systems, where they are not the entire reason the game exists in the first place, is that they exhibit one or other of the standard failure modes:
  • too weak for the player to bother using against the AI
  • so strong that the meta revolves around the sabotage mechanics even though it wasn't intended to
  • too annoying to allow the AI to use against the player
Paradox GSGs are no exception to this.

Fair point, though I think a situation of 'Defense beats Offense beats Sabotage beats defense' could be reached.

The thing is as you say, you cant add espionage in after the start so this would be a complete rebalance of war and combat with espionage included (which, we kinda do need tbh).
 
If static defences can stop you, then static defences + a fleet can't lose.

That's called "Sabotage Starbase" as it should work, instead of the worthless action it currently is. You successfully hack their starbase and disable, depending on how critical your success role was, a various number of fleet power. 10%, 40%, 80% whatever. And if it's your starbase, you give players "Counterintelligence" actions to keep other empires out of your systems. You know, what Espionage is suppose to freaking do. That system is the answer to the problem, it just requires Espionage to actually be well designed, instead of the worthless system it currently is. You can also create a new Anti-Starbase weapon type for Colossi that can crack the starbase, like the Ottoman Cannons that finally broke Constantinople. If the defender cant get clear shots through your screening shield fleet in time then the base goes bye bye. This is not a very difficult issue to come up with solutions to. Either the developers simply can't come up with any, never bothered to do so, or deliberately wants the game to have poor strategy elements.
 
  • 4
  • 2
  • 1Like
Reactions:
That system is the answer to the problem, it just requires Espionage to actually be well designed, instead of the worthless system it currently is.
I have never seen this mythical "well-designed" Sabotage system.
 
  • 4
  • 1Like
Reactions:
All this said, I do think Starbases aren't totally useless. If you pick unyielding, build a stratcom center, and fill a bastion with ion cannons, you can get a Starbase with 100k+ish fleet power, which is more than good enough for repelling attacks by AI empires so long as they're not FEs or Crisis, which is the sort of passive border guarding I imagine is the current intent. Even in the very late game on GA I've never seen one of the AIs manage to build a single fleet with more than 60-70k power... excluding some federation fleets... and obviously stacks... which the AI has been using more recently I feel... but still decent for a passive sort of defense against border intrusion/inner-empire guarding at chokepoints.

........no actually I do see the point, this is *after* speccing into Unyielding, really the former should be the default and Unyielding should let you make real obstacles for enemies. *eyes military planet stations*
 
  • 2Like
  • 1
Reactions:
All this said, I do think Starbases aren't totally useless. If you pick unyielding, build a stratcom center, and fill a bastion with ion cannons, you can get a Starbase with 100k+ish fleet power, which is more than good enough for repelling attacks by AI empires so long as they're not FEs or Crisis, which is the sort of passive border guarding I imagine is the current intent. Even in the very late game on GA I've never seen one of the AIs manage to build a single fleet with more than 60-70k power... excluding some federation fleets... and obviously stacks... which the AI has been using more recently I feel... but still decent for a passive sort of defense against border intrusion/inner-empire guarding at chokepoints.

........no actually I do see the point, this is *after* speccing into Unyielding, really the former should be the default and Unyielding should let you make real obstacles for enemies. *eyes military planet stations*
I'd like new/ancient toys to help with this: https://forum.paradoxplaza.com/foru...nihilator-cannon.1494003/page-2#post-27898012
 
  • 1Like
Reactions:
If static defences can stop you, then static defences + a fleet can't lose.
Just come back with a bigger fleet, or multiple fleets.
I'd love to encounter the AI making a single last stand and be on or close to a 1-1 ratio, rather than eight battles where I have 10x their fleet power in each battle.
 
  • 2
  • 1Like
Reactions:
Just come back with a bigger fleet, or multiple fleets.
I'd love to encounter the AI making a single last stand and be on or close to a 1-1 ratio, rather than eight battles where I have 10x their fleet power in each battle.

Precisely. If you can't, or refuse, to go around. Then guess what? you gotta break down the wall, with every ship in your empire all at once if necessary. Whatever it takes to crack the wall. If that does not work, then you try to lure the enemy fleets away with a feint attack from another direction. You know, actual tactics. Instead of just rolling ones face across the keyboard clicking "attacking system" over and over until the wars over.
 
  • 1Like
  • 1
Reactions:
Yeah, just got bitten by this. Reinforcements were built at a Starbase FOUR SYSTEMS AWAY within my own territory. FIVE YEAR reinforcement time. Actual travel time: 4 months.

Yea my entire campaign has been plagued with this. Making me debate really hard about not even finishing. It's shocking how broken the pathfinding is right now.
 
  • 2Like
Reactions:
Tossing my 5 coins into whole fleet reinforcement stuff.

As I see the discussion, whole lot of workarounds tied to the issue about how to make auto reinforcement equivalent to manual reinforcement. But. What if we need not a tonload of workarounds, but redesign of fleet reinforcement and production mechanics at all?
 
I've heard there's some kind of bug in the current version that greatly increases MIA time because the pathfinding is broken.
That's correct. For 3.1, we did changes to when bypasses were allowed to be used by the pathfinder. This fixed a few bugs and uncovered (rather than caused) some existing ones.

In many cases related to MIA, we checked the distance of the path to estimate how long a fleet should go MIA for. This breaks when they start using bypasses as they have massive distances, but doesn't actually take that long to travel through.

Most (all?) issues of this type should be addressed for 3.2.
 
  • 10Like
  • 4
  • 2Love
Reactions:
That's correct. For 3.1, we did changes to when bypasses were allowed to be used by the pathfinder. This fixed a few bugs and uncovered (rather than caused) some existing ones.

In many cases related to MIA, we checked the distance of the path to estimate how long a fleet should go MIA for. This breaks when they start using bypasses as they have massive distances, but doesn't actually take that long to travel through.

Most (all?) issues of this type should be addressed for 3.2.
There's nothing like the feeling of realising there were two wrongs.
 
Hello, I was wondering if there’s any possibility of seeing the before and after changes statistics for a lower grade machine, say a i3 10th gen *cough* I imagine your changes will be felt more on slower machines?
 
Hello, I was wondering if there’s any possibility of seeing the before and after changes statistics for a lower grade machine, say a i3 10th gen *cough* I imagine your changes will be felt more on slower machines?
Numbers will be roughly same, expressed in relative values. In UX terms, your slowdown (when your rig can't calculate galaxy fast enough to fit into timeframe) will start at X% further in total number of pops that exists in galaxy than before.
 
All this said, I do think Starbases aren't totally useless.
My main complaint with Starbases is that they are very effective against GA AI and not at all effective against a good player. AI is being worked on though, so maybe Starbases will be rebalanced as a result of this.

As for defense platforms, I don't know why there was a nerf, they tend to get wiped out in an alpha strike after getting off like one shot.

I always assumed they were deliberately resource inefficient. You see you are getting attacked unexpectedly and rush to build up some patch work defenses with what's on hand. They aren't good and are very wasteful, but you pump them out quick and they can slightly delay an enemy fleet to buy time for your forces to muster. (That seems to be the intent, but I don't think they are very good at this role.)

Either way, the actual way the game wants you to do defense is with a fortress world or habitat with a planetary shield, an FTL inhibitor, and a lot of ground forces.

That situation basically requires a colossus weapon to crack through. And it works against all Crisis except unbidden (who ignore FTL inhibitors for some reason).
 
My main complaint with Starbases is that they are very effective against GA AI and not at all effective against a good player. AI is being worked on though, so maybe Starbases will be rebalanced as a result of this.

As for defense platforms, I don't know why there was a nerf, they tend to get wiped out in an alpha strike after getting off like one shot.

I always assumed they were deliberately resource inefficient. You see you are getting attacked unexpectedly and rush to build up some patch work defenses with what's on hand. They aren't good and are very wasteful, but you pump them out quick and they can slightly delay an enemy fleet to buy time for your forces to muster. (That seems to be the intent, but I don't think they are very good at this role.)

Either way, the actual way the game wants you to do defense is with a fortress world or habitat with a planetary shield, an FTL inhibitor, and a lot of ground forces.

That situation basically requires a colossus weapon to crack through. And it works against all Crisis except unbidden (who ignore FTL inhibitors for some reason).

That is a form of defense, but I would personally argue forever delaying/ stalling someone, with no chance of driving them off, is not truly defense. Fortress Worlds would realistically have giant guns to shoot at ships in orbit, for example. And don't tell me that concept does not exist in Stellaris, because I've seen plenty of Precursor "Orbital Defense Guns" modifiers on planets over the years. Except those don't do anything, they just reduce damage taken, which is dumb.

This idea of defense simply being "Welp, guess we'll just sit here and get bombed forever until either we die or a fleet shows up, and take no action ourselves" is ridiculous.
 
  • 2Like
Reactions:
That is a form of defense, but I would personally argue forever delaying/ stalling someone, with no chance of driving them off, is not truly defense. Fortress Worlds would realistically have giant guns to shoot at ships in orbit, for example. And don't tell me that concept does not exist in Stellaris, because I've seen plenty of Precursor "Orbital Defense Guns" modifiers on planets over the years. Except those don't do anything, they just reduce damage taken, which is dumb.

This idea of defense simply being "Welp, guess we'll just sit here and get bombed forever until either we die or a fleet shows up, and take no action ourselves" is ridiculous.
This is a good point and something they should add. We have "orbital defense shield" as a building; devs, why not an equivalent "anti-orbit cannon" building that shoots back at bombarding fleets? Seems like a useful tool that would make sense and be implemented without many new systems being required.
 
  • 3
Reactions:
That's called "Sabotage Starbase" as it should work, instead of the worthless action it currently is. You successfully hack their starbase and disable, depending on how critical your success role was, a various number of fleet power. 10%, 40%, 80% whatever. And if it's your starbase, you give players "Counterintelligence" actions to keep other empires out of your systems. You know, what Espionage is suppose to freaking do. That system is the answer to the problem, it just requires Espionage to actually be well designed, instead of the worthless system it currently is.
There's a mod that improves this aspect of espionage. As well as espionage in general.