Anatomy of a Game: Simulation Optimization

  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
Performance issues can be a pain. At my work we have a whole team dedicated to measuring the microcontroller ressources and managing large scale optimisations. You really don't want an ECU in a car take too long to calculate something, because if that happens you'll see christmas lightning in your dashboard. ;)

I assume that in the official released build you deactivate the whole measuring methods? Because in the end the measuring and analyzing also eats up ressources which are only useful if someone actually looks at them.
 
  • 1Like
Reactions:
low key this is awesome
 
  • 2
Reactions:
Yet more pointless optimisation. Game runs better than any other paradox game already, yet right now there's not enough content to even want to play into late-game and see the benefits of the optimisation. People complaining about low fps must be running FX's or 2nd gen intel CPU's
 
  • 8
Reactions:
Yet more pointless optimisation. Game runs better than any other paradox game already, yet right now there's not enough content to even want to play into late-game and see the benefits of the optimisation.
More optimization is always good. You see, it's not only about how it makes the game run in its current state, but what kinds of future improvements it makes feasible. :)



People complaining about low fps must be running FX's or 2nd gen intel CPU's
Maybe, but so what? This is a grand strategy game. It makes sense to make it more playable even for people with low-end machines. :)
 
  • 1
Reactions:
More optimization is always good. You see, it's not only about how it makes the game run in its current state, but what kinds of future improvements it makes feasible. :)




Maybe, but so what? This is a grand strategy game. It makes sense to make it more playable even for people with low-end machines. :)
There's a difference between a low-end machine, and a 10-year old CPU that costs less than the game.
 
People complaining about low fps must be running FX's or 2nd gen intel CPU's
As we have just been shown, just having an HDD instead of SSD can make a tremendous difference to the worst, and most users don't have much choice in the HDD/SSD matter. My notebook is of 2014 and has HDD, and I have every right to expect CK3 to lag... a bit... freaking... less (just watch my segment on ModCon to see what I am talking about - and it's a cut-for-public version, original was forty minutes long and included a whole minute of fighting with lags non-stop as well as many shorter sessions of the same; granted, that segment was filmed with Zoom and Discord working in the background but that shouldn't be enough of a reason).
yet right now there's not enough content to even want to play into late-game and see the benefits of the optimisation
I can't really agree even if it were true that the benefits only matter for late-game (which is false, see above). CK3 has much more content than base CK2 had (and I don't even mean "had back when it was launched", which is trivial, I mean "if you launch CK2 now with no DLCs"), and base CK2 has been fun enough to want to play till 1453 for many people.
 
  • 1Like
Reactions:
There's a difference between a low-end machine, and a 10-year old CPU that costs less than the game.
I didn't think you were being quite so literal.
 
I feel like I'm taking crazy pills here, with people talking about how CK3 is the fastest PDX game. For me, it's the slowest, and by quite a big margin. But with many people saying it runs very well for them, I suspect my performance woes are the result of a rare bug with certain hardware combinations.

CK3, for me, sometimes takes up to a minute for a single in-game day, the game constantly pauses and unpauses while running, and the framerate swings *wildly*. I'm pretty confident in saying even the M&T mod for EU4 runs faster. And before someone jumps on me, saying I'm using decades old hardware, I have a 11700K, 32GB RAM and a 2070S. It's not about mods either, I'm currently playing modded but I had the exact same issues with a completely vanilla game, that I've reinstalled from scratch.

I wonder if the team is aware of this issue? Or is this how the game runs for everyone. Anyhow, I've attached one VTune Hotspot analysis summary (without callstacks) and one Parallelism analysis, if it's useful.

I suspect it has to do with NPC calculations stalling other cores. If so, I'd really like to see more concurrency here, or, if that's not possible, consider using SIMD here? If you already are, I would love to see AVX512, considering the only consumer CPU that has it (Rocket Lake) doesn't have any AVX512-related clock slowdown.
 

Attachments

  • hotspot.PNG
    hotspot.PNG
    95,4 KB · Views: 0
  • parallelism.PNG
    parallelism.PNG
    106,8 KB · Views: 0
  • 1
Reactions:
I feel like I'm taking crazy pills here, with people talking about how CK3 is the fastest PDX game. For me, it's the slowest, and by quite a big margin. But with many people saying it runs very well for them, I suspect my performance woes are the result of a rare bug with certain hardware combinations.

CK3, for me, sometimes takes up to a minute for a single in-game day, the game constantly pauses and unpauses while running, and the framerate swings *wildly*. I'm pretty confident in saying even the M&T mod for EU4 runs faster. And before someone jumps on me, saying I'm using decades old hardware, I have a 11700K, 32GB RAM and a 2070S. It's not about mods either, I'm currently playing modded but I had the exact same issues with a completely vanilla game, that I've reinstalled from scratch.

I wonder if the team is aware of this issue? Or is this how the game runs for everyone. Anyhow, I've attached one VTune Hotspot analysis summary (without callstacks) and one Parallelism analysis, if it's useful.

I suspect it has to do with NPC calculations stalling other cores. If so, I'd really like to see more concurrency here, or, if that's not possible, consider using SIMD here? If you already are, I would love to see AVX512, considering the only consumer CPU that has it (Rocket Lake) doesn't have any AVX512-related clock slowdown.
That is crazy slow and sounds definitely like some edge case or hardware combo issue because not even our min spec machines run it that slowly, on the min or recommended specs its one of the best performing (if not the best) of our games. On speed 5 (uncapped tick rate) for good hardware its to the point of unplayable in how fast it runs the ticks.

"NPC calculations stalling cores" doesn't really mean anything, that isn't how our parallelism works. Nor is throwing some SIMD at something gonna really magically solve anything.

To get a minute for a single day you basically have to be running the application logic single threaded, as then all the threading in the game logic is basically not done if there are no worker threads making it a hell of a lot slower, do you have any command line overwriting the worker threads count? As it shows you've got a very low average amount of active cores (which makes sense given how slow it is) but I can't really think of a reason in our code for it to do that outside of the thread count override since then its just the main thread trying to do the daily tick instead of all the other cores joining in.

Are you maybe on some weird broken OS version to the point it can't detect number of threads to make workers for? That'd be super unlikely though as we do not do anything weird there and sounds like other games run fine for you.

I'd be super interested if you made a thread in the tech support sub-forum and included as much info as you can from the game (a fresh install if possible and mods disabled), some stuff off the top of my head to include:
  1. Full hardware specs
  2. All logs from starting the game
  3. Separately all logs after running the game for a bit
  4. Settings file info
  5. Is it installed on an SSD or HDD
  6. How long it takes to launch
  7. Any command line arguments used
Does this happen on all versions I wonder? If you use the steam beta feature to roll back to a previous version of CK3 do they all have the same issue?
 
  • 3Like
  • 1
Reactions:
"NPC calculations stalling cores" doesn't really mean anything, that isn't how our parallelism works. Nor is throwing some SIMD at something gonna really magically solve anything.
What I meant by that, is, if NPC-related logic is necessarily ran on two threads, SIMD would help wrap up those calculations faster, and therefore leave more room for "more concurrent" code to run. But, if it runs on more than two threads, then you're correct, it won't help for this specific issue.

I'd be super interested if you made a thread in the tech support sub-forum and included as much info as you can from the game (a fresh install if possible and mods disabled), some stuff off the top of my head to include:
Sure, I can do that. Do you want me to tag you on that thread?
 
What I meant by that, is, if NPC-related logic is necessarily ran on two threads, SIMD would help wrap up those calculations faster, and therefore leave more room for "more concurrent" code to run. But, if it runs on more than two threads, then you're correct, it won't help for this specific issue.


Sure, I can do that. Do you want me to tag you on that thread?
Yeah we use all the threads we can, I’d suggest checking out last weeks anatomy of a game on how our game state works it explains our threading for game logic a bit actually

Please do tag me yes! I won’t respond or look in right away (got other work scheduled) but will give us a complete report for the future!
 
  • 3Like
Reactions:
Hello everyone and welcome back to Anatomy of a Game, I’m Matthew a programmer on Crusader Kings 3 and today I am going to be talking about some optimisations I’ve done for 1.5 to put some of the higher level examples we’ve shown off in the previous two posts in a more concrete context.

...

And that is all for this week folks! Next week I will be digging into the script system as a whole and how that works to let our Content Designers and modding community make all the cool stuff they do which fills our game up with fun things!
Wow. I v rarely post (or log in, for that matter), but I always read these dev diaries, and I felt like I should comment to say how utterly fascinating I found this. It's v rare to get a glimpse behind the curtain to see the amount of work--and even more than that, the *type* of work--that goes into games these days. More of this, please!
 
  • 1Love
Reactions: