• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #149 - Technical improvements

Hi everyone, this is Moah. I’m the tech lead on Stellaris and today I’m here to talk about the free 2.3 "Wolfe" update that will be arriving together with Ancient Relics, and what it brings to the table in terms of tech.

Stellaris is going 64 bits.
People have been clamoring for this for a while now, and various factors have led us to finally do this for this patch. I should temper your expectations though: while many have claimed that this would be a miracle cure for all their issues with Stellaris, the reality is somewhat more tame.

What does it mean?
The one solid benefit is that Stellaris is no longer limited to 4gb of memory, and won’t crash anymore in situations where it was reaching that limit. For people who play on huge galaxies, with many empires, many mods or well into 3000s, this will be a boon.

In terms of performance, though, it doesn’t change much. Without drowning you in technical details, let’s just say that some things go faster because you handle more data at once, some things go slower because you have more data to handle. In the end, our measurements have shown no perceptible difference.

Finally, the last effect of switching to 64 bits is that the game will no longer playable on 32 bits computers or OSes. We don’t think this will affect many people, but there you have it.


What about Performance?
I know that’s everyone’s favourite question, so let’s do our best to talk about it. First, let me dispel some notions floating around in various forums: Stellaris does use multithreading, and we’re always on the lookout for new things to thread. In fact between 2.2.0 and 2.2.7, a huge effort was made to thread jobs and pops, and it’s one of the main drivers of performance improvement between these version.

Pops and jobs are indeed what’s consuming most of our CPU time nowadays. We’ve improved on that by reducing the amount of jobs each pop evaluate. We’ve also found other areas where we were doing too much work, and cut on:
  • Ships calculating their daily regeneration when they’re at full health
  • Off-screen icons being updated
  • Uninhabitable planets doing the same evaluations as populated planets
Why do these seemingly pointless things happen? Well, we generally focus on getting gameplay up and working quickly so that our content designers can iterate quickly, and sometimes things fall through the cracks. Some of these systems are also quite complex and the scale of the new code is not so easily apparent. Sometimes, not limiting the number of targets is good enough because you’re not doing much but then, months later, someone adds more calculations or the number of objects explodes for unrelated reasons, and suddenly you’ve got a performance issue.

Modifiers
One thing that sets Stellaris apart from other PDS title is how much we use (or abuse) modifiers. Everything is a modifier. Modifiers are modified by other modifiers themselves modified by other modifiers, and sometimes by themselves. It’s quite hard to follow, and leads to every value being able to change at any time without your noticing.

“Why don’t you just compute jobs when a new one appears?” has often been asked around these parts. Well, a short answer to that is it’s really hard to know when a new job appears. You can get jobs from any modifier to: country, planet, pops. Each of these can get modifiers from ethics, traditions, perks, events, buildings, jobs, country, planets, pop, technology, etc.

Until now we were trying to calculate modifiers manually, forced to follow the chain in its entirety: when you recompute a country modifier, you then calculate their planets modifiers, and then each planet would recalculate their pops modifiers. Some of our freezes were just that tangled ball of yarn trying to sort itself out.

NexRiPkna2utTqAzF9H0DEjOCwHVsI4EejYO-vMQMh6QwUB-_uP7dXmpjkwXzOOKoiwDqkSzd9tlLmN3DlFN2R06A62od6XxWm8xh99XRDfRFRP3vVj42GBIaDaXSK7jjyKdS39b

This is our modifier flow charts. It’s not quite up to date, but gives you an idea of the complexity of the system (Unpolished because it’s a dev tool, and not made for the article).

No More!
For 2.3 “Wolfe” we have switched to a system of modifier nodes, where each node register what node they follow, and is recalculated when used, following the chain itself. We have modifiers that are more up to date, and calculated only when needed. This also reduces the number of pointless recalculations.

This system has shown remarkable promise, and cut the number of “big freezes” happening around the game (notably after loading, for example). It has some issues, but as we continue working with it, it’ll get better and help both with performance and our programmers’ sanity.

So, what’s the verdict?
In our tests, 2.3 “Wolfe” is between 10% and 30% faster than 2.2.7 right now. Hopefully it’ll stay that way until release, but the nature of the beast is that some of these optimizations break things and fixing the issues negate them, so we can’t promise anything.

IuIGuQ4cXPvjCEMWG_AowiNIFXhzpsPIcphmCVJD79vQqVMqUeZCqCoVfDlWDNZ3YNkAScYAJh2ebft947YsqoOhG7A_4pNBWxjZ6L9se5lkEEImNYZ4uOpTMWj-amEiwSYdirpd


Measurements provided by @sabrenity , using detailed info from the beta build. It’s worth noting the “SHIPS_SERIAL” purple line has since been eliminated.

AI
Another forum favorite, we have done some improvements to the AI. First, with @Glavius ’s permission, we’ve used his job weights to improve general AI job distribution. We’ve also done the usual pass of polish and improvements, and of course taught the AI how to use all our new features.


What else is new?
We’re also getting a new crash reporter that will send your crash report as soon as they happen rather than next time you start the game. We’ve improved our non-steam network stack for connectivity issues, etc.


All right, enough of my yammering. This has turned into a GRRM length novel, and even though there are many more areas we could cover, we’ll just turn this for your perusal.
 
Last edited:
  • 1
Reactions:
AI
Another forum favorite, we have done some improvements to the AI. First, with @/Glavius’s permission, we’ve used his job weights to improve general AI job distribution. We’ve also done the usual pass of polish and improvements, and of course taught the AI how to use all our new features.

If I could make a future AI request, could the AI learn to hard counter the mid/endgame crisis, and if there's multiple ones, choose the closest, and/or strongest?
 
The last 32-bit CPU was 17 years ago. Can those computers even run Stellaris? And there's literally no reason for anyone with a newer CPU to have a 32-bit OS. In short, anyone who's affected by this kinda deserves it.

Actually in my country and other parts of the world we don't change our computer every time something new is out, we change them when they broke. So by personal experience i assure you people is still using 32-bit CPU more than 17 years old. I could make the change to windows 7 last year. For economics problems as well as import troubles and pay rates if a family or even one person buys something is something they will use a long time, a very long time.

I am not, in any way, complaining about the change. As i said the more performance issues fixed the better. Is just that for some people getting a crappy computer for US/EU standards is getting a really good one down here. I just wanted to say this for my personal experience and i already have made that. My catharsis is complete, do as you want whit my two cents :)
 
The move to 64 bits is the occasion to get rid of many overflow bugs that plague the game. Either by using bigger variables, or by setting caps to prevent the values to overflow and become negative.
Examples:
Weapons range overflow prevents whole fleets from firing.
Research points overflow prevents your empire from building anything.
etc.


Yes! Please the tech overflow bug will be automatically solved with this update?
All the overflow issues hamstring late games I reach into
 
Stellaris is going 64 bits.
People have been clamoring for this for a while now, and various factors have led us to finally do this for this patch. I should temper your expectations though: while many have claimed that this would be a miracle cure for all their issues with Stellaris, the reality is somewhat more tame.

I know they're not necessarily directly connected (as you can use larger int sizes in 32 bit) but will the many overflow bugs in the game be addressed?
 
The last 32-bit CPU was 17 years ago. Can those computers even run Stellaris? And there's literally no reason for anyone with a newer CPU to have a 32-bit OS. In short, anyone who's affected by this kinda deserves it.
Actually, many businesses have programs that are extremely difficult to update that they might be legally obligated to keep running for any variety of reasons that cant be ran on a 64 bit OS. For example, many hospitals in the US have systems that are still running MS 95 or older because the program cant run on newer OSes, including often things like their databases. They legally cant update these without going through an extensive list of things they need to make sure can be brought over, or they could risk potential HIPAA violations. And that is if they have the right to modify the program. Most dont. And that of course assumes the hardware they use can be updated to newer OSes and programs. There are pieces of medical equipment still in use that use assembler. Fucking Assembler. Updating any of these costs money. A lot of hospitals dont have enough to spare however.
Yes! Please the tech overflow bug will be automatically solved with this update?
All the overflow issues hamstring late games I reach into

This wont necessarily solve it. It just gives it a lot more breathing room before it can cause a problem.
 
All of this looks great, i really hope for it to works as you say. My computer is not exactly state of the art, so any performance improvement is welcome.

Thanks for addressing this issues :)

It will affect more people than you think.

Sometimes people just need to get on with the times. 64 but has been around for 20+ years, Windows 10 64bit was a free upgrade for 18 months
 
Thank you very much for your work. I agree with people here that providing devs with time to pay off the technical debt and to improve the architecture of the game by decoupling the components would improve stability and create more room for future content improvements.

In the context of performance I have a question however - do you consider the removal of turrets from ships? It's been a topic on forums, and even wallpapers of ships released on artstation are without turrets (this one for the reason that it is more beautiful this way).
I believe that it can improve performance and that the community is quite united about the ugliness of those turrets.
 
Yes! Please the tech overflow bug will be automatically solved with this update?
All the overflow issues hamstring late games I reach into
As a generic answer: 64 bit won't magically fix the overflows we have, but many of those have been fixed independantly by the team.
 
Actually, many businesses have programs that are extremely difficult to update that they might be legally obligated to keep running for any variety of reasons that cant be ran on a 64 bit OS. For example, many hospitals in the US have systems that are still running MS 95 or older because the program cant run on newer OSes, including often things like their databases. They legally cant update these without going through an extensive list of things they need to make sure can be brought over, or they could risk potential HIPAA violations. And that is if they have the right to modify the program. Most dont. And that of course assumes the hardware they use can be updated to newer OSes and programs. There are pieces of medical equipment still in use that use assembler. Fucking Assembler. Updating any of these costs money. A lot of hospitals dont have enough to spare however.
You are playing Stellaris on medical equipment?
 
The last 32-bit CPU was 17 years ago. Can those computers even run Stellaris? And there's literally no reason for anyone with a newer CPU to have a 32-bit OS. In short, anyone who's affected by this kinda deserves it.

You might have a Windows 7 32b disc from years ago and not want to spend money on a new OS disc...or install Linux. Okay, yeah. There is no reason not to have a 64b OS. Never mind.

Actually in my country and other parts of the world we don't change our computer every time something new is out, we change them when they broke. So by personal experience i assure you people is still using 32-bit CPU more than 17 years old. I could make the change to windows 7 last year. For economics problems as well as import troubles and pay rates if a family or even one person buys something is something they will use a long time, a very long time.

I am not, in any way, complaining about the change. As i said the more performance issues fixed the better. Is just that for some people getting a crappy computer for US/EU standards is getting a really good one down here. I just wanted to say this for my personal experience and i already have made that. My catharsis is complete, do as you want whit my two cents :)

Considering the performance problems people with new computers are having, can those 17 year old machines run Stellaris at anything faster than a crawl? I think that's the real point, not whether or not people are still using old machines.

Will there be a modding dev diary before release?
I for one am concerned about the changes to how modifiers are being calculated. That sounds like a prime spot for modder headache.
 
As a generic answer: 64 bit won't magically fix the overflows we have, but many of those have been fixed independantly by the team.
But it would facilitate moving from short to long integers, and from single to double precision floats right? By itself, using a 64b register won't fix overflows, but moving variables from 32b to 64b should correct? And a 64b executable is a first step.
 
Actually in my country and other parts of the world we don't change our computer every time something new is out, we change them when they broke. So by personal experience i assure you people is still using 32-bit CPU more than 17 years old. I could make the change to windows 7 last year. For economics problems as well as import troubles and pay rates if a family or even one person buys something is something they will use a long time, a very long time.

I am not, in any way, complaining about the change. As i said the more performance issues fixed the better. Is just that for some people getting a crappy computer for US/EU standards is getting a really good one down here. I just wanted to say this for my personal experience and i already have made that. My catharsis is complete, do as you want whit my two cents :)

All of those cpus you refer to are far under minimum spec for the game so it’s a bit of a ridiculous argument to make.

Absolutely everyone who meets the requirements has a 64bit cpu. Everyone.
 
Hi everyone, this is Moah. I’m the tech lead on Stellaris and today I’m here to talk about the free 2.3 "Wolfe" update that will be arriving together with Ancient Relics, and what it brings to the table in terms of tech.

Stellaris is going 64 bits.
People have been clamoring for this for a while now, and various factors have led us to finally do this for this patch. I should temper your expectations though: while many have claimed that this would be a miracle cure for all their issues with Stellaris, the reality is somewhat more tame.

What does it mean?
The one solid benefit is that Stellaris is no longer limited to 4gb of memory, and won’t crash anymore in situations where it was reaching that limit. For people who play on huge galaxies, with many empires, many mods or well into 3000s, this will be a boon.

In terms of performance, though, it doesn’t change much. Without drowning you in technical details, let’s just say that some things go faster because you handle more data at once, some things go slower because you have more data to handle. In the end, our measurements have shown no perceptible difference.

Finally, the last effect of switching to 64 bits is that the game will no longer playable on 32 bits computers or OSes. We don’t think this will affect many people, but there you have it.


What about Performance?
I know that’s everyone’s favourite question, so let’s do our best to talk about it. First, let me dispel some notions floating around in various forums: Stellaris does use multithreading, and we’re always on the lookout for new things to thread. In fact between 2.2.0 and 2.2.7, a huge effort was made to thread jobs and pops, and it’s one of the main drivers of performance improvement between these version.

Pops and jobs are indeed what’s consuming most of our CPU time nowadays. We’ve improved on that by reducing the amount of jobs each pop evaluate. We’ve also found other areas where we were doing too much work, and cut on:
  • Ships calculating their daily regeneration when they’re at full health
  • Off-screen icons being updated
  • Uninhabitable planets doing the same evaluations as populated planets
Why do these seemingly pointless things happen? Well, we generally focus on getting gameplay up and working quickly so that our content designers can iterate quickly, and sometimes things fall through the cracks. Some of these systems are also quite complex and the scale of the new code is not so easily apparent. Sometimes, not limiting the number of targets is good enough because you’re not doing much but then, months later, someone adds more calculations or the number of objects explodes for unrelated reasons, and suddenly you’ve got a performance issue.

Modifiers
One thing that sets Stellaris apart from other PDS title is how much we use (or abuse) modifiers. Everything is a modifier. Modifiers are modified by other modifiers themselves modified by other modifiers, and sometimes by themselves. It’s quite hard to follow, and leads to every value being able to change at any time without your noticing.

“Why don’t you just compute jobs when a new one appears?” has often been asked around these parts. Well, a short answer to that is it’s really hard to know when a new job appears. You can get jobs from any modifier to: country, planet, pops. Each of these can get modifiers from ethics, traditions, perks, events, buildings, jobs, country, planets, pop, technology, etc.

Until now we were trying to calculate modifiers manually, forced to follow the chain in its entirety: when you recompute a country modifier, you then calculate their planets modifiers, and then each planet would recalculate their pops modifiers. Some of our freezes were just that tangled ball of yarn trying to sort itself out.

NexRiPkna2utTqAzF9H0DEjOCwHVsI4EejYO-vMQMh6QwUB-_uP7dXmpjkwXzOOKoiwDqkSzd9tlLmN3DlFN2R06A62od6XxWm8xh99XRDfRFRP3vVj42GBIaDaXSK7jjyKdS39b

This is our modifier flow charts. It’s not quite up to date, but gives you an idea of the complexity of the system (Unpolished because it’s a dev tool, and not made for the article).

No More!
For 2.3 “Wolfe” we have switched to a system of modifier nodes, where each node register what node they follow, and is recalculated when used, following the chain itself. We have modifiers that are more up to date, and calculated only when needed. This also reduces the number of pointless recalculations.

This system has shown remarkable promise, and cut the number of “big freezes” happening around the game (notably after loading, for example). It has some issues, but as we continue working with it, it’ll get better and help both with performance and our programmers’ sanity.

So, what’s the verdict?
In our tests, 2.3 “Wolfe” is between 10% and 30% faster than 2.2.7 right now. Hopefully it’ll stay that way until release, but the nature of the beast is that some of these optimizations break things and fixing the issues negate them, so we can’t promise anything.

IuIGuQ4cXPvjCEMWG_AowiNIFXhzpsPIcphmCVJD79vQqVMqUeZCqCoVfDlWDNZ3YNkAScYAJh2ebft947YsqoOhG7A_4pNBWxjZ6L9se5lkEEImNYZ4uOpTMWj-amEiwSYdirpd


Measurements provided by @sabrenity , using detailed info from the beta build. It’s worth noting the “SHIPS_SERIAL” purple line has since been eliminated.

AI
Another forum favorite, we have done some improvements to the AI. First, with @Glavius ’s permission, we’ve used his job weights to improve general AI job distribution. We’ve also done the usual pass of polish and improvements, and of course taught the AI how to use all our new features.


What else is new?
We’re also getting a new crash reporter that will send your crash report as soon as they happen rather than next time you start the game. We’ve improved our non-steam network stack for connectivity issues, etc.


All right, enough of my yammering. This has turned into a GRRM length novel, and even though there are many more areas we could cover, we’ll just turn this for your perusal.

I enjoy under the hood looks like this. It's interesting to see how "just make it faster" goes from wish, through the tunnel of programming, into reality.
 
Performance improvements are always good. Stellaris is an awesome game, and the only 2 recurring problems I ever had with it were late game lag and dumb AI. Glavius fixed the latter, and you seem to be taking more steps (or at least explaining better how you're taking more steps) towards fixing the former. I know it won't solve every issue like a magic wand, but every little bit helps, one way or another.
 
Just the kind of dev diary I'd like to see more often, regardless of how unimpressive it may seem to you devs. Excellent work, and a sexy graph (and flowchart) to boot! Please, may we have some more?