• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
Hi everyone! I am Caligula, one of Stellaris’ Content Designers, which means that I do a variety of tasks based around narrative writing and scripting - “scripting” being our term for doing things that is somewhat similar to programming, but without changing the source code. In other words, I do what modders do (though I have the significant advantage of also being able to peek into the source code and change it around when needed). Every Content Designer has their niche, and mine is that when a particularly complicated system needs to be scripted in (or, more frequently, is giving some sort of trouble - the War in Heaven still gives me nightmares...), I step into the breach.

Now, we have a lot of exciting stuff to show off in the weeks and months to come, but for today, inspired by some questions that were asked after the last dev diary, I’m going to be writing about the technical side of scripting for modders and aspiring modders, specifically with an eye on what can cause performance problems and how to avoid making bad scripts.

The Stellaris scripting language is a very powerful tool, and a lot can be done with it, but first of all, a note of caution: just because something is possible, does not mean it should be done. I can’t really stress this enough, because (and I speak from experience here) this attitude will almost certainly end up causing both performance issues and unreadable scripts that you will not be able to disentangle six months later when you realise some part of it is broken. Though it should be borne in mind that doing something in code is, by definition, faster: in code, you can check a single function and be done with it, but if you want it to be accessible through script, there’s a fair few necessary functions it has to go through before you get to checking your function (turning the line of script into a code command, checking whether it’s used in the right scope, etc etc) - hence why some things are hardcoded, and also why hacky solutions to problems can end up being quite bad. So, the first question to consider is, should I really be doing this?

But who am I kidding, I’m speaking to modders here, so of course you will do it :D So without further ado...

What causes performance issues?

Every time you run a check or execute an effect, this will take a very tiny amount of your computer’s processing power. With a few exceptions that should be used sparingly (I’ll get to those later), this is totally fine and is needed to do anything at all. It is when the check is repeated often, over lots of objects, that problems happen. In practice, this usually means pops are the cause, though running something across all planets in the galaxy is also a pretty bad idea.

As a first step, when possible, it is a good idea to control when your script is run. The best way to do this is by setting where events are fired and using on_actions (or firing events from decisions and the like) wherever possible, instead of mean time to happen or, even worse, just setting an event to try and fire every day. If a degree of randomness is needed, one could also fire a hidden event via, say, a yearly pulse and then firing the actual event you want with a random delay (for an example, check out event action.220).

Of course, not everything is events, and unfortunately, in Stellaris, a lot of stuff is done with pops! Job weights and other triggers in the jobs files, in particular, have been shown to cause issues in the past. As a rule of thumb, even if you can do super cool stuff, it is a bad idea to do any too complicated script pyromania on jobs. For example, if you were to make a job weight dependent on there being no other pops on the planet that are unemployed (using planet = { any_owned_pop = { is_unemployed = yes } }), then you are doing a regular check on every pop on the planet that then checks every other pop on the planet, i.e. pops squared. Once you reach the late game, this is pretty much guaranteed to cause issues.

So what can be done?

Avoiding nested loops, making sure your events are fired appropriately and avoiding pops when possible will get you some way, but we can do better than that. Here is my list of advice for optimising scripts:

Always use the most appropriate scope

Say you want to check something on the leader of the current country’s federation. One could, theoretically, do it this way:
Code:
        any_country = {
            is_in_federation_with = root
            is_federation_leader = yes
            <my_triggers_here> = yes
        }

This'll run through all countries in the game and see whether they are in the same federation as you, including the space amoeba country and the vile Pasharti (and who would want them in a federation?). That’s a bunch that are definitely irrelevant. So a better check would be to do it this way:
Code:
        any_federation_ally = {
            is_federation_leader = yes
            <my_triggers_here> = yes
        }

In code terms, this means that the game going from the country to the federation and then grabbing a list of its members, excluding the current country, and checking the triggers against them. So, that’s obviously going to be fewer checks. However, the best version would be this:
Code:
        federation.leader = {
            <my_triggers_here> = yes
        }

That version would go straight to the federation and from there straight to its leader in the code, with as little as possible script to code conversion needed and no need to check triggers against any countries to get there. It also happens to be the most readable (readability and better performance very often correlate…).

So in this case, the game would check around 50 countries first the first version, 5 for the second and 1 for the third - not bad for some optimisations! Using a similar logic, it is always better to use something that isn’t checking all objects in the galaxy (esp. all pops or all planets) if at all possible but rather a filtered list, e.g. any_planet_within_border instead of any_planet = { solar_system.owner = { is_same_value = prevprev } } (you laugh, but I’ve seen it). And, indeed, one can almost always check any_owned_fleet instead of any_owned_ship.

Another important improvement we added in 2.6 was any_owned_species, which can replace many any_owned_pop checks (specifically the ones that check for traits and so on of the pop) and mean that way, way fewer objects have to be checked (in a xenophobic empire, it could be single figures for any_owned_species and thousands for any_owned_pop).

Sometimes you can avoid scopes completely

On a similar note, if you can check something without doing things with scopes, that’s always going to be better. So, if one wants to check whether a planet has more than two pops working as miners, one could do this two ways:
Code:
        count_owned_pop = {
            count > 2
            limit = {
                has_job = miner
            }
        }
Code:
        num_assigned_jobs = {
            job = miner
            value >= 2
        }

The former will check each pop on the planet and see whether it has the miner job, and then see whether the number that do is higher than 2. The latter will check a cached number that the game has already calculated and see if it is more than 2, which is much quicker to do.

Some things are just expensive

Not every check or effect is equal. Checking a flag or a value is generally pretty simple, and changing it is usually not much more complicated. If, however, the game has to recalculate stuff, then it will take longer, because it’s not just looking up a number it already knows. Creating new stuff is also more expensive, both because it’s doing something somewhat complicated (the create_species effect is, I kid you not, more than 600 lines of C++ code...), and because it’ll probably have to recalculate all sorts of values once this is done. It can be a bit tricky to know which triggers and effects are going to be bad, but as a rule, these cases are what you should look out for:
  • Anything where you are creating a new scope e.g. create_country, create_species, modify_species
  • Anything that needs you to calculate or recalculate pathfinding (e.g. can_access_system trigger, creating new hyperlanes, especially creating new systems)
  • Anything that calculates pops (changing around pop jobs on a planet, for instance)

If it must be done...

Sometimes, bad things must be done. In these cases, it is best to still use the not so great things with precision. When the game is checking triggers e.g. for an event, it’ll generally stop checking them at the first point something returns false (I’m told this is called “short-circuit evaluation”), so you’ll want to do something like this:
Code:
    trigger = {
        has_country_flag = flag_that_narrows_things_down_loads
        <something really horrible here>
    }

I recently did something like this to the refugee pop effect. It was previously a little bit insane (see 01_scripted_triggers_refugees.txt for the full horror). In total, it would check a variation of the following up to eight times:
Code:
        any_relation = {
            is_country_type = default
            has_communications = prev #relations include countries that have made first contact but not established comms
            NOT = { has_policy_flag = refugees_not_allowed }
            prevprev = { #this ensures Pop scope, as root will not always be pop scope
                OR = { 
                    has_citizenship_type = { type = citizenship_full country = prev }
                    has_citizenship_type = { type = citizenship_caste_system country = prev }
                    AND = {
                        has_citizenship_type = { type = citizenship_limited country = prev }
                        has_citizenship_type = { type = citizenship_caste_system_limited country = prev }
                        prev = { has_policy_flag = refugees_allowed }
                    }
                }
            }
            any_owned_planet = {
                is_under_colonization = no
                is_controlled_by = owner
                has_orbital_bombardment = no
            }
        }

Where it varied was simply the last any_owned_planet: It would try and find a relation with a really good planet for the pop to live on, then a pretty good, then a pretty decent, and then finally settle for just any old planet. Which is, obviously, pretty inefficient, since the list of countries that welcome refugees does not change between each of the 8 times you check it. My way of avoiding this - and making the script way more readable, whilst I was at it - was to set a flag before any of the checks, like this:
Code:
        every_relation = {
            limit = {
                has_any_habitability = yes #bare minimum for being a refugee destination
            }
            set_country_flag = valid_refugee_destination_for_@event_target:refugee_pop
        }

The checks then simply had to be “does the country have the flag, if yes, does it have a good enough planet”:
Code:
has_good_habitability_and_housing = {
    has_country_flag = valid_refugee_destination_for_@event_target:refugee_pop
    any_owned_planet = {
        habitability = { who = event_target:refugee_pop value >= 0.7 }
        free_housing >= 1
        is_under_colonization = no
        is_controlled_by = owner
        has_orbital_bombardment = no                                
    }
}

One can also similarly use if-limits and elses (and, even better, switches when possible - those are the best for performance) in triggers to narrow down the checks down and make things far more readable whilst you are at it. I recently went through the species rights files and redid the allow triggers for sanity’s sake:

(Before)
Code:
        custom_tooltip = {
            fail_text = MACHINE_SPECIES_NOT_MACHINE
            OR = {
                has_trait = trait_mechanical
                has_trait = trait_machine_unit
                from = { has_valid_civic = civic_machine_assimilator }
            }
        }
        custom_tooltip = {
            fail_text = ASSIMILATOR_SPECIES_NOT_CYBORG
            OR = {
                NOT = { from = { has_valid_civic = civic_machine_assimilator } }
                AND = {
                    OR = {
                        has_trait = trait_cybernetic
                        has_trait = trait_machine_unit
                        has_trait = trait_mechanical
                    }
                    from = { has_valid_civic = civic_machine_assimilator }
                }
            }
        }

(After)
Code:
        if = {
            limit = {
                from = { NOT = { has_valid_civic = civic_machine_assimilator } }
            }
            custom_tooltip = {
                fail_text = MACHINE_SPECIES_NOT_MACHINE
                OR = {
                    has_trait = trait_mechanical
                    has_trait = trait_machine_unit
                }
            }
        }
        else = {
            custom_tooltip = {
                fail_text = ASSIMILATOR_SPECIES_NOT_CYBORG
                OR = {
                    has_trait = trait_cybernetic
                    has_trait = trait_machine_unit
                    has_trait = trait_mechanical
                }
            }
        }

The second version will be more efficient, since it is only checking e.g. whether the species has the mechanical trait or whether the country has the assimilator civic once instead of twice, and also, the triggers in the second custom tooltip aren’t obscenely weird anymore. (I also removed all the NANDs, because they broke my brain).

Happy New Year

I can’t write this dev diary without telling you about the “Happy New Year” bug. Basically, we were playing dev MP on a reasonably large galaxy and reached reasonably late into the game, and since we were all working remotely on wildly varying computers and internet connection speeds, the performance was perhaps a tad sluggish, but still acceptable for the most part. Then, suddenly, we noticed huge lag spikes - 20 seconds and more - on the 1st of January. So noticeable were these spikes that we began wishing each other a Happy New Year each time the game froze!

It just so happened that the onset of this lag coincided with several large empires deciding to become synthetic and starting to assimilating their empires. Now, assimilation falls into the category of things that are done in script that maybe, in hindsight, should probably not have been done that way… and works by firing an event for each assimilating country every 1st of January. This event in turn fired an event for each of their planets that selected a bunch of pops on the planet and used modify_species on each of them at least once, but sometimes up to four times. This added up to a fairly significant performance hog!

After trying various solutions, it turned out the best way to fix this was to first go through every_owned_species from the country scope, check whether this species should be assimilated, and if so use modify_species to create the species it would assimilate to, setting a species flag that pointed to the species it was being assimilated from. Then, instead of creating a new species for every pop that was assimilated, the scripts were rewritten to find the already-created species that the pop should become, and simply use change_species on it. The result is still unreadable script (I will spare your eyes and not post it here), but in my tests it reduced the yearly tick by over 50%, thanks to the complicated effect (modify_species) being run as seldom as possible.

---

That’s it for me, for now! I’m guessing this was a bit of a drier dev diary than usual for most of you, but hopefully it was interesting nonetheless :) As a final note, I suspect that the more intrepid among you could call me out on bits of script in the base game that don’t quite live up to these guidelines. Please, feel free to do so, because there are few things more satisfying in this profession than taking something really horrible and making something less horrible out of it!
 
At that point, you might as well make all pops an abstract concept and just have it be an increasing variable that determines everything else through just maths and percentages. If you're going to make some pops "fake", you might as well make all of them "fake" and use a system that doesn't track individuals beyond "X pops, of Y species with Z modifier" all as simple sets of numbers.

true. my thought was that pops are so integral and that people like pops. at least at the start of the game. later when you have 10k.. not so much. Personally, while i liked pops before I could see pops go the way of the dodo. that "compromise" was just meant as appeasement to the pop wanting majority. And as is said. brainstorming.
 
Last edited:
What about the horrible mod load times before you even start a game? It has dramatically lengthen since 2.8.

There is an issue with users who are using many (40+) mods. It does seem to be combining these mods into one mega-mod helps if the game "fails" to start. (the game is actually still loading in the background, most times). We're currently investigating whether this is a game issue or a launcher issue. Hope to know more soon.
 
  • 2Like
  • 1
Reactions:
well, my loading times didn't change on game start from 2.7 to 2.8, i'm using round 15 mods but i'm on a HDD so...i hoped for a little faster gamestart but i'm ok with that, just want to tell you if that may help find the problem xD
 
PDX scripting language and generally DSL is a dead idea.
The interactive fiction literate-programming DSL Inform 7 (compiled to bytecode for the Glulx and Z special-purpose virtual machines) is a much better language for writing parser-based interactive fiction in than any general-purpose language. (There are specific slices of the IF design space where Inform 6 would be easier and/or more performant, but in general Inform 7 is the better choice.)

The visual novel DSL Ren'Py is a much better language for writing visual novels in than any general-purpose language. (It embeds Python for doing complex things, but the core scripting is very definitely a DSL.)

Choice of Games's Choicescript DSL is probably a better tool for writing choose-your-own-adventure games in than any general-purpose language.

But go on, please tell me how DSLs are a dead idea.
 
I know this is a bit of an older post, and I'm performing a bit of necromancy here, but this is SUPER HELPFUL. Thank you Caligula for elucidating us modders on how to improve the performance of our scripts. I was performing a lot of checks with a lot of flags in my script, and as the script began to get really inflated, I started wondering "Am I making this more efficient or LESS efficient?" yeah. It was definitely LESS efficient. Anyway thanks again!
 
  • 1
  • 1
Reactions:
well, i would like to see more posts that gives modders tips and tricks how to make theyr mods better, more efficient and also some informations about how some scripts work, how to mod some difficult parts and also very important, how to avoid overwriting stuff
 
  • 2
Reactions:
I myself am using around 23 mods or thereabouts; and for the life of me, it consistently fails to launch- hey, I never even *found* where to download into the machine account mod 2.8 to begin with.
 
I myself am using around 23 mods or thereabouts; and for the life of me, it consistently fails to launch- hey, I never even *found* where to download into the machine account mod 2.8 to begin with.

Customer support will gladly help you fix this problem. Go to support.paradoxplaza.com and open a ticket. :)
 
Your new profile pictures keep tripping me up XD

I was really tempted to ask to see if we could keep the Jeff portrait hidden just for me, but I felt that would be greedy. :D