Anatomy of a Game: Startup and Loading

  • Victoria 3 - Sign up now!

    The journey begins, sign up now and get a special in-game item when the game is released.


  • Crusader Kings III Available Now!

    The realm rejoices as Paradox Interactive announces the launch of Crusader Kings III, the latest entry in the publisher’s grand strategy role-playing game franchise. Advisors may now jockey for positions of influence and adversaries should save their schemes for another day, because on this day Crusader Kings III can be purchased on Steam, the Paradox Store, and other major online retailers.


    Real Strategy Requires Cunning
  • Crusader Kings II Expansion Subscription

    Subscribe to the CK II Expansion and enjoy unlimited access to 13 major expansions and more!


Alertcriminal

Recruit
112 Badges
Nov 23, 2015
1
3
  • A Game of Dwarves
  • Arsenal of Democracy
  • Cities in Motion
  • Cities in Motion 2
  • Crusader Kings II
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Darkest Hour
  • Dungeonland
  • Europa Universalis III
  • Divine Wind
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Heir to the Throne
  • Impire
  • Knights of Pen and Paper +1 Edition
  • Lead and Gold
  • The Kings Crusade
  • Magicka
  • March of the Eagles
  • Europa Universalis IV: Res Publica
  • Semper Fi
  • Sengoku
  • Ship Simulator Extremes
  • Sword of the Stars II
  • Supreme Ruler 2020
  • The Showdown Effect
  • Victoria 2
  • Victoria 2: Heart of Darkness
  • War of the Vikings
  • Europa Universalis IV
  • Warlock: Master of the Arcane
  • War of the Roses
  • Europa Universalis IV: El Dorado
  • Magicka: Wizard Wars Founder Wizard
  • Crusader Kings II: Horse Lords
  • Crusader Kings II: Conclave
  • Imperator: Rome Deluxe Edition
  • Age of Wonders III
  • Cities: Skylines Deluxe Edition
  • Crusader Kings III
  • Steel Division: Normandy 44
  • Prison Architect
  • Gettysburg
Nice to get a deeper insight into how things are working behind the scenes, really enjoyed the read, thanks for taking the time to write this. :)
 
  • 2
  • 1Like
Reactions:

x4077

Lt. General
Sep 16, 2020
1.617
3.450
To take a modern example of that, the elevator rides in Mass Effect 1 where you had wonderful party banter in places like the Citadel were just loading screens hidden with narrative exposition on top, but even when running the game on modern hardware it would not let you skip them.
In the recently released Mass Effect Legendary Edition remaster they added a skip button during those scenes now as modern hardware can load everything quickly, you can of course still opt into hearing the great banter between your squad mates.
Oh if this were only true...

I gave up playing Mass Effect the first time it was released because of the inability to skip through the cut scenes. I decided to try it out again with the re-release and made it about an hour before uninstalling and returning it for the same damn reason.

On Topic: What was the rationale behind packaging all the audio files into proprietary .BANK files and not providing an abstraction configuration file somewhere so that individual sound effects could be enabled/disabled via the end user?

CK2 was great about this as the sound effects were each in their own separate files, so that if one or two particular sounds were particularly annoying the user could do something about it. When CK3 1.1 came out with the constant ambient siege effect it was driving me so nuts that I eventually just ended up renaming the entire ambient .BANK file to get rid of it, along with all the other ambient sounds of course.
 
Last edited:
  • 1Angry
  • 1
Reactions:

dabrain2

Corporal
16 Badges
Apr 7, 2004
39
1
  • Europa Universalis IV
  • Hearts of Iron IV Sign-up
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: Together for Victory
  • Hearts of Iron IV: Death or Dishonor
  • Age of Wonders III
  • Hearts of Iron IV: Expansion Pass
  • Hearts of Iron IV: Expansion Pass
  • Imperator: Rome Deluxe Edition
  • Imperator: Rome
  • Hearts of Iron IV: Expansion Pass
  • Imperator: Rome Sign Up
  • Hearts of Iron IV: La Resistance
  • Imperator: Rome - Magna Graecia
  • Crusader Kings III
  • Battle for Bosporus
Hello everyone! My name is Matthew and I am a programmer on CK3, welcome to the first in a series of more technical focuses posts called Anatomy of a Game.

In this series over the summer we are going to try and pull back the curtain of game development a bit and go into more detail on how our games actually work.

What happens when you launch up the game? How do we make sure to load DLCs and mods? What actually goes on in a single day of gameplay? What makes the AI decide to brutally invade you?

I hope to answer, or at the very least shed some light on, some of these questions and more over the course of this series.

These posts are not a replacement for our more traditional Dev Diaries, and if you are looking for information or teasers related to upcoming content then this is not the thing for you. But if you are interested in the background of what makes games work or are an aspiring game developer then these should prove interesting to you!

Game Setup​

In today’s post I am going to begin at well… the beginning! When you launch the game and load in to pick your character what is it that we are actually doing?
It might sound like a simple question, as well obviously we load everything right? But how do we do that in a way that feels responsive and as fast as possible? What stuff do we actually load vs deal with later? How do mods and DLC files play into overriding things?

At a high level the flow of starting the game is:
  1. Start some base functionality in the engine in order: file system, logging, threading task system
  2. Load a library called SDL, which is a common library for accessing hardware in games
  3. Load a ton of stuff for the engine to do its job, many as background tasks on threads
  4. Create the window for the game
  5. Load an even larger ton of stuff for the actual game itself
  6. Game is now loaded and you can start playing
So why does step 1 do it sub-steps in that specific order? Its purely a dependency that makes sense if you think about it, a lot of systems do logging so but that of course needs files so we load the file system first, and we want to load a lot of things in background threaded tasks but that system is going to log so we need to do it after the logging and file system.

This is a very high level set of steps and glosses over a lot of details of course, I will be glossing over numerous details in this but feel free to ask any specific questions. The bulk of interesting things happens in steps 3 and 5, they are the biggest part of our startup loading and where most of any optimization time goes.

Why optimize load times you might ask? Well of course for you the end user whilst our load screens and main themes are great you’d probably rather be playing the game.
For us, well we start the game up a lot during any given day. And every second spent waiting for the game to load is just wasted time and money. When we run the game we also do so without the compiler having done its heavier optimisations on our code, so ours is even slower than yours.

It is of course not the core of our optimization times, we work much more on daily tick speed and framerate. But it is an area we keep in mind especially for our iteration speed payoffs in the day to day development.

How does one even optimize a lot of these things? It's generally split between making more efficient use of CPU time and reducing the time spent waiting.

For the CPU time that is a couple of things: multi-threading more work so more of the CPU is being used, optimize your lower level algorithms you use, and finally you can get really close to the hardware to optimize your CPU branch misprediction and memory cache misses.

For waiting the goal is just to do less of it. Waiting is whenever your CPU has to wait to get access to a resource, wait for some threaded work to actually be done at its deadline, and even things like reading data from the disk since that does take in which your code is doing nothing itself.

The one I am going to go into more detail here initially is higher level multithreading usage, as our loading of data on the game side is the one I’ve spent time optimizing at the beginning of the year and most of that was solved by introducing more multithreading capabilities and optimising things that were thread safe but not thread efficient.

Something to look at​

Our very first goal with loading is that we want to get a load screen up ASAP so that you know the game hasn’t crashed and burned on you, so we show the main splash screen there.
View attachment 740061
This wonderful logo is covering up us trying to load everything we can and make you not think it is all crashing and burning. After we have it up we start loading some necessary game components up front like the mouse textures and the bare minimum UI files like how our text should look to show the proper load screen.

Another thing we try to load ASAP is the music database so we can get that main theme playing, cause just staring at a load screen in silence is less fun than having our composer’s wonderful music to tide you over the loading. Even though ironically one of our goals is to make you hear as little of that main theme as possible. Sorry Andreas…

Incidentally in other games if you see an un-skippable cutscene like someone opening a door to walk through, or one that you can only skip part of the way through, then in reality that is just a very pretty loading screen hiding a lot of work being done under the hood.

Thankfully games these days account for the fact that hardware gets better and lets you skip a lot of them once the loading is done, but if you go back to older games that was not a thought so many long scenes hiding a load cannot be skipped.

To take a modern example of that, the elevator rides in Mass Effect 1 where you had wonderful party banter in places like the Citadel were just loading screens hidden with narrative exposition on top, but even when running the game on modern hardware it would not let you skip them.
In the recently released Mass Effect Legendary Edition remaster they added a skip button during those scenes now as modern hardware can load everything quickly, you can of course still opt into hearing the great banter between your squad mates.

We’ve loaded a lot of stuff multithreaded until now, but from here on out everything we’re loading is threaded in the game side. And we load a lot now, things like the rest of files for the UI, data for our 3D entities, the map data, our game logic files, and then the front end and its UIs.

All of this loading is done whilst the main thread keeps making sure we intermittently render the load screen and poll the operating system for input events and promptly ignore them all.

The latter being because Windows will give you the dreaded application not responding pop up if you fail to poll the events it provides, since we are loading we cannot really do anything meaningful with any inputs so we just take and ignore them to make Windows sure we haven’t secretly crashed.

View attachment 740062

Also worth noting that for a good amount of stuff we don’t actually have it loaded even by the time you are at the main menu, because the average user is going to maybe decide what save they want, be a bit slow on clicking something etc. and all of that does not require us to have everything loaded. So we aim to continue loading other lower priority things in the background until we really truly need it.

One example of that is our history database, it is very large but also something we know that we do not need to get to the main menu and even then we only need if we are starting a new game. As saves contain their own history from the play time. So we keep that loading in the background giving us some nice extra free time to the point where it's probably loaded even if you do start a new game eventually anyway.

Database Loading​

So that has been a lot of words about the things we load and some reasons behind the priorities. Now I am going to pivot into a more specific example, loading our game and mod files into the various databases.

So how we get DLC and mod files working in the same level as the game is actually quite straightforward. For our file system we use an integrated and modified version of PhysicsFS, or PhysFS for short, which works very nicely by letting you mount multiple directories under one logical hierarchy you can then search through to find individual files or multiple files in the hierarchy to load from.

During startup when we set up our file system we mount our base directories for the core game, and then when we load DLCs and mods we look in the user directory and check what is enabled and then mount those directories into the same path. So searching for a folder like “common/landed_titles” we will look in both the game install directory and the directories of enabled downloaded mods and treat them as the same from the code’s perspective when we load that data into the landed title database.

So how do we actually load these databases? In the currently released version of CK3 we load them all on one of our background tasks and one by one load them in order, with the order having support for defining dependencies as some systems need to reference another. Which for a long time was “good enough”, but as we’ve added more and more scripted systems and more and more content in sheer volume to load that started to show its flaws.

It was the biggest amount of time on all our threaded loading tasks and dominated by a lot meaning that we were waiting on just that one chunk of work even after the other things were done, and in terms of optimisation opportunities it was very good because in theory the databases can be formalised in a very strict layout and we then know what we can and cannot load simultaneously.

To note this idea of loading databases in parallel would not have been doable without the work that one of our other programmers, @MatRopert the Tech Lead on HoI4, did when he was on Stellaris which improved our PhysFS file system to operate a lot better in a heavily threaded environment. As if file operations entirely were a threading bottleneck then we’ve had no gains by trying to do this anyway, that is a common problem that one bottleneck hides many others. I’d recommend checking out his blog post on it here, though note it is aimed at programmers more directly than this post.

So what was it that I did here? First step was formalising our database dependencies into a proper graph object in code that I could then operate on. Be that dumping it to disk as an image to cry at our dependencies, running automated tests on it to ensure that changes and additions do not break everything, or doing the actual loading of it all so we can play the game.

Setting up the dependency graph is straightforward, every database’s code can be analysed to see if it needs a strict dependency or not. Generally that means if the reading of the script file directly needs to then reference another database, such as reading in a key and looking it up, then it needs a strict relation to be added.

Here is the dependency graph I captured recently for just our logic, no interface or databases with icons present, and it's already quite the monster.

View attachment 740065

The loading of this graph is where we get our performance gains, since we know exactly what databases depend on what we can load in parallel databases that do NOT depend on each other, which is the bulk of databases. Then when something is loaded it can notify things dependent on it that it has been completed and if it was the last thing they needed they can start their loading and so on.

Anything with threading is always easier said than done, you’ve got to make sure your operations really don’t go around competing with each other or introduce exclusive ownership or atomic operations where necessary so things stay synchronized. And when doing so make sure that the speed ups you are doing don’t then get wiped out by all of the synchronization that you are doing.

So of course then queue up a lot of discovering of hidden dependencies I had to formalise, fixing dreaded circular dependencies, plenty of threading problems, optimizing a few more systems, and doing some iterations to squeeze all the performance we could out of it and we managed to get some nice gains!

So for some actual numbers of these changes, and others done by myself and @Meneth:
  • Internal debug builds this section took over 20 seconds to load and now it's down to under 4 seconds.
  • In the release builds it took this section down from about 7 seconds down to under a second.
So it won’t suddenly make the game load instantly and deprive you of the wonderful main theme, but it definitely should save you some time and saves us a lot of time we can then spend actually making the game instead of browsing twitter for memes or making XKCD posts proud…

compiling.png


History into reality​

Once we’re at the main menu we can do the final part of getting you into the game, loading a save or starting a new game. Of which both are pretty similar hence I am bundling them together.

Loading a save is at a high level a simple one, we create our base game state and then simply run through your save game file and read in different sections which fill in different data for every part of the game. Our saves, and all files in our games really, operate on a fairly simple syntax of data that we read in.

The saves can get pretty big, since our simulations can store a lot of numbers and data, so for a lot of things we either try to not write it to save games if it's a default value or we just cache it in game internally so it never appears in saves.

An example of this is the full modifiers that apply to a character from all sources, we do not actually write those in our normal save games because it's just so much data and is something we can rebuild entirely from the rest of the save. We only write them when performing an out of sync check and want to compare them directly to make sure none of the values differ or that our caches are incorrect.

Starting a new game is actually really just the process of making a new save based on the start date you pick and then loading that save. If you select a bookmark character then we fast track you through the lobby to that character when loading, but if you select play as anyone then really we just load up the save game and put the pick a character lobby up to you.

The initial save generation is mostly just a lot of reading history files and then filling in default data. We also do a lot of semi-randomised startup content so that not every detail of every play session is always identical, though nothing on the level of something like Stellaris where the entire starting scenario changes every time since we do want to keep a fairly consistent historical background to the playthroughs.

View attachment 740067

Reading and loading all 15 MB and the near 600 files...

If you are a modder you may notice that we've got a lot less small history files these days and instead bunch them into bigger ones. This is due to the fact that on Windows the system for opening a file is not as fast as on something like Linux or OSX so we instead aim to open fewer but bigger files. Especially since we try to do a lot of our loading more aggressively in parallel now its a boon to not be waiting on the file system, hence less tiny little files these days.

We made it!​

So we’ve made it into the game! Nice! Time to actually play it, or if you’re me doing optimisations, close it, change some code, and launch it up again a few times to measure more...

I hope you’ve enjoyed this little (can one consider like 8 pages in google docs little?) dive into some of the backing of the game and technical details of it. I’ve tried to stay fairly high level because most of you are likely not software engineers, but please do give feedback on if it's too high level or getting a bit too technical so I can adapt for the future posts!

I wanted to do a series like this because game development is a bit of a black box knowledge wise to our end users compared to other forms of entertainment like books, television or music. Everyone knows roughly what goes into writing a book or how difficult it can be to play a musical instrument, in many countries we have to study such things in school.

But game development is not like that, it is the largest entertainment industry in the world but most of the consumers don’t really know what goes into it leading to misconceptions, and I think the more we can try to reduce some of the mystique of it the easier time we’ll have being able to communicate with our fans overall. And hopefully get a few people interested in maybe pursuing a career in it.

So that is all for this week folks! Feel free to ask questions here of any technical level and I will try my best to answer them!
Well with all that talk there should be more optimization of the loading times of the games even on a i9 and a m2 ssd rated at 3.5gb/sec the game takes very long to load when compared to other games.
 
  • 2
Reactions:

Seclouds

First Lieutenant
13 Badges
Mar 14, 2014
255
5
  • Crusader Kings II
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Knights of Pen and Paper +1 Edition
  • Magicka
  • Naval War: Arctic Circle
  • Semper Fi
  • The Showdown Effect
  • War of the Roses
  • Mount & Blade: Warband
  • Crusader Kings III Referal
  • Crusader Kings II: The Old Gods
The launcher is always circling the log in and if it didn't finished the circling, I can't launch the game.
And when I'm in the game, it's always disconnect or kick/ lost connection from the server?
It said I need to fill my email and pw and log in again in the game, after the launcher had already successful log in.
It's really annoy.
Do you guys need to collect the information about how many in-game characters generated by the out of marriage bang?
So that we need to always log in and play the game.
Ck2 doesn't have this problem. please fix it. There are some complain about this in steam community.
 

HydranMenace

Recruit
58 Badges
Oct 19, 2019
6
14
  • Crusader Kings II
  • Pillars of Eternity
  • Shadowrun: Hong Kong
  • Stellaris: Synthetic Dawn
  • Stellaris: Galaxy Edition
  • Hearts of Iron IV: Cadet
  • Crusader Kings II: Reapers Due
  • Europa Universalis IV: Rights of Man
  • Stellaris: Digital Anniversary Edition
  • Stellaris: Leviathans Story Pack
  • Hearts of Iron IV: Together for Victory
  • Crusader Kings II: Monks and Mystics
  • Stellaris - Path to Destruction bundle
  • Cities: Skylines - Mass Transit
  • Hearts of Iron IV: Death or Dishonor
  • Stellaris
  • Age of Wonders III
  • Cities: Skylines - Green Cities
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Humanoids Species Pack
  • Stellaris: Apocalypse
  • Stellaris: Distant Stars
  • Shadowrun Returns
  • Shadowrun: Dragonfall
  • Stellaris: Megacorp
  • Crusader Kings II: Holy Fury
  • Hearts of Iron IV: Expansion Pass
  • Crusader Kings III
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sword of Islam
  • Europa Universalis IV
  • Battle for Bosporus
  • Crusader Kings III: Royal Edition
  • Stellaris: Galaxy Edition
  • Stellaris: Federations
  • Hearts of Iron IV: La Resistance
  • Stellaris: Lithoids
  • Stellaris: Ancient Relics
  • Cities: Skylines
  • Mount & Blade: Warband
  • Crusader Kings II: Way of Life
  • Cities: Skylines - After Dark
  • Crusader Kings II: Conclave
  • Cities: Skylines - Snowfall
  • Cities: Skylines Deluxe Edition
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
Like others said, interesting read. I don't need to know most of it since I'm just a gamer not a programmer or modder but still enjoyable to go through. Thanks.
 

Dayvit78

Colonel
49 Badges
Apr 11, 2017
1.061
1.391
  • Europa Universalis 4: Emperor
  • Imperator: Rome Deluxe Edition
  • Europa Universalis IV: Common Sense
  • Europa Universalis IV: Third Rome
  • Europa Universalis IV: Rights of Man
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV: Mandate of Heaven
  • Imperator: Rome
  • Crusader Kings III
  • Imperator: Rome - Magna Graecia
  • Crusader Kings II: Rajas of India
  • Europa Universalis IV
  • Crusader Kings II: Jade Dragon
  • Europa Universalis IV: Pre-order
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Mare Nostrum
  • Crusader Kings II: The Republic
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Rule Britannia
  • Europa Universalis IV: Res Publica
  • Crusader Kings II: Horse Lords
  • Crusader Kings II: Conclave
  • Crusader Kings II: Reapers Due
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: Golden Century
  • Crusader Kings II: Monks and Mystics
  • Europa Universalis IV: Cossacks
  • Europa Universalis IV: Cradle of Civilization
  • Europa Universalis IV: Dharma
  • Crusader Kings II: Holy Fury
  • Imperator: Rome Sign Up
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Art of War
  • Crusader Kings II
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sword of Islam
  • Hearts of Iron III Collection
  • Darkest Hour
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • For the Motherland
  • Semper Fi
  • Arsenal of Democracy
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Victoria 2
Very nice to hear the background of this!
I just got a new laptop and my loading time went from like 5 minutes to under 30 seconds. The practical impact is that I play this game more often because I know I can jump in jump out very quickly. You're right that faster loading times are much enjoyable.
 

blackninja9939

Experienced Programmer - Crusader Kings 3
76 Badges
Aug 28, 2013
2.338
5.134
  • Crusader Kings III
  • Stellaris: Federations
  • Battle for Bosporus
  • Stellaris: Necroids
  • Europa Universalis 4: Emperor
  • Europa Universalis IV
  • Crusader Kings III: Royal Edition
  • Crusader Kings II
  • Crusader Kings II: Holy Fury
  • Imperator: Rome - Magna Graecia
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: The Old Gods
  • Europa Universalis IV: Rights of Man
  • Europa Universalis IV: Cradle of Civilization
  • Stellaris: Synthetic Dawn
  • Surviving Mars
  • BATTLETECH
  • Europa Universalis IV: Mandate of Heaven
  • Crusader Kings II: Monks and Mystics
  • Tyranny: Archon Edition
  • Europa Universalis IV: Rule Britannia
  • Crusader Kings II: Reapers Due
  • Hearts of Iron IV: Colonel
  • Stellaris Sign-up
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Apocalypse
  • Stellaris: Lithoids
  • Stellaris: Distant Stars
  • Europa Universalis IV: Dharma
  • Shadowrun Returns
  • Imperator: Rome Deluxe Edition
  • Prison Architect
  • Imperator: Rome Sign Up
  • Stellaris: Ancient Relics
  • Age of Wonders: Planetfall
  • Crusader Kings II: Conclave
  • Crusader Kings II: The Republic
  • Victoria 2
  • Cities: Skylines
  • Europa Universalis IV: El Dorado
  • Crusader Kings II: Way of Life
  • Stellaris
  • Mount & Blade: Warband
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: Sword of Islam
Oh if this were only true...

I gave up playing Mass Effect the first time it was released because of the inability to skip through the cut scenes. I decided to try it out again with the re-release and made it about an hour before uninstalling and returning it for the same damn reason.

On Topic: What was the rationale behind packaging all the audio files into proprietary .BANK files and not providing an abstraction configuration file somewhere so that individual sound effects could be enabled/disabled via the end user?

CK2 was great about this as the sound effects were each in their own separate files, so that if one or two particular sounds were particularly annoying the user could do something about it. When CK3 1.1 came out with the constant ambient siege effect it was driving me so nuts that I eventually just ended up renaming the entire ambient .BANK file to get rid of it, along with all the other ambient sounds of course.
Huh can’t say I’ve had the same experience with my replays of the legendary edition, I’ve skipped plenty of scenes on redos.

The Bank files are because we use a much more sophisticated and better third party audio library called FMOD, our old audio used to be way way too simplistic for what our audio designers and composters wanted.

Since we're talking about the loading phase : what's up with the "Initializing Game" icon on the top left that looks like it should turn, but doesn't ? =)
I’m like 80% sure that’s just cause we’re not updating the animation system whilst we’re loading on the front end or something, is very minor hence not really looked into it.
Well with all that talk there should be more optimization of the loading times of the games even on a i9 and a m2 ssd rated at 3.5gb/sec the game takes very long to load when compared to other games.
There should always be more optimisation for everything, I think that is obvious. It’s a matter of priority.
With a good processor and ssd I’d be surprised if it takes that long to load, is sub 60 seconds on my pretty decent but by no means amazing computer and that’s in full debug mode. Release is faster by a lot.
HDDs are a lot slower and we’re aware of that, though it’s again a matter of prioritisation to work on.
The launcher is always circling the log in and if it didn't finished the circling, I can't launch the game.
And when I'm in the game, it's always disconnect or kick/ lost connection from the server?
It said I need to fill my email and pw and log in again in the game, after the launcher had already successful log in.
It's really annoy.
Do you guys need to collect the information about how many in-game characters generated by the out of marriage bang?
So that we need to always log in and play the game.
Ck2 doesn't have this problem. please fix it. There are some complain about this in steam community.
This is entirely unrelated to this dev diary and I’m not really sure what you’re talking about, make a bug report or open a support ticket with more info as they are the right places to report issues.
 
  • 4Like
  • 2
  • 1
Reactions:

Chatoustikmou

Captain
82 Badges
May 21, 2016
464
746
  • Crusader Kings II: Charlemagne
  • Stellaris: Synthetic Dawn
  • Europa Universalis IV: Third Rome
  • Europa Universalis IV: Mandate of Heaven
  • Cities: Skylines - Mass Transit
  • Stellaris - Path to Destruction bundle
  • Tyranny: Archon Edition
  • Europa Universalis IV: Rights of Man
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Crusader Kings II: Jade Dragon
  • Europa Universalis IV: Mare Nostrum
  • Cities: Skylines - Snowfall
  • Europa Universalis IV: Cossacks
  • Cities: Skylines - After Dark
  • Magicka 2
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II
  • Crusader Kings II: Sons of Abraham
  • Cities: Skylines Deluxe Edition
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sword of Islam
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Art of War
  • Cities: Skylines - Natural Disasters
  • Stellaris: Digital Anniversary Edition
  • Stellaris: Leviathans Story Pack
  • Hearts of Iron IV: Cadet
  • Crusader Kings II: Monks and Mystics
  • Surviving Mars
  • Hearts of Iron IV: Death or Dishonor
  • Cities: Skylines - Green Cities
  • Crusader Kings II: Reapers Due
  • Hearts of Iron IV: Field Marshal
  • Hearts of Iron IV: Colonel
  • Europa Universalis IV
  • Stellaris: Galaxy Edition
  • Crusader Kings II: Conclave
  • Stellaris: Necroids
  • Hearts of Iron IV: Expansion Pass
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Cities: Skylines
  • Pillars of Eternity
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: Pre-order
I'm like 80% sure that’s just cause we’re not updating the animation system whilst we’re loading on the front end or something, is very minor hence not really looked into it.

"very minor"? It's literally unplayable! :eek:

Jokes aside, this was a great read. I hope somewhere in this series, there'll be the same breakdown of what happens when you close the game. The many (harmless) errors thrown + the fact after I've rebooted the game a number of times it stops exiting properly and requires me to quit Steam to force quit CK3 has made me curious :)
 
  • 1Haha
Reactions:

wyldmage

Corporal
60 Badges
Jan 7, 2013
37
18
  • Crusader Kings II
  • Warlock 2: Wrath of the Nagas
  • Warlock 2: The Exiled
  • Victoria: Revolutions
  • Stellaris - Path to Destruction bundle
  • Crusader Kings II: Way of Life
  • Pillars of Eternity
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Horse Lords
  • Crusader Kings II: Conclave
  • Stellaris
  • Crusader Kings II: Reapers Due
  • Stellaris: Leviathans Story Pack
  • Crusader Kings II: Monks and Mystics
  • Europa Universalis IV: El Dorado
  • BATTLETECH
  • Stellaris: Synthetic Dawn
  • Age of Wonders III
  • Age of Wonders
  • Crusader Kings II: Jade Dragon
  • Stellaris: Apocalypse
  • BATTLETECH - Backer
  • Cities: Skylines - Parklife
  • Prison Architect
  • Shadowrun Returns
  • Shadowrun: Dragonfall
  • Stellaris: Megacorp
  • Crusader Kings II: Holy Fury
  • Crusader Kings III
  • Crusader Kings III: Royal Edition
  • Age of Wonders: Planetfall Sign Up
  • Europa Universalis IV
  • Stellaris: Lithoids
  • Age of Wonders: Planetfall - Revelations
  • Age of Wonders: Planetfall
  • Victoria 2
  • Stellaris: Federations
  • Magicka: Wizard Wars Founder Wizard
  • Warlock: Master of the Arcane
  • Cities: Skylines
  • Stellaris: Ancient Relics
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • King Arthur II
  • Europa Universalis IV: Res Publica
Thanks for posting this!

Was an interesting read, and definitely good to know how seriously you guys take those loading times.

One of the stories I tell people most is about visual optimization. My old PC was running a Nvidia 960 GT video card, which struggled to punch out high video settings for most good looking games, but did fine if I turned down things like water reflections, dynamic shadows, etc. The games still looked good to me overall.
However, Ark was terrible. The shadows were poorly done no matter what settings I used, and if I ran the game at much beyond 'medium' settings, I'd drop below 20 fps (I still drop below 20fps occasionally playing that, but only when starting at huge hordes of tamed dinos, for example).

But, one day, a new game came out - Fallout 4. I installed it and fired it up. "Recommended Ultra Settings". I scoffed - my video card doesn't handle that. But I figured I'd let it load, and adjust it down as needed. Surprise, though, I got out of the starting vault, and the game looked absolutely gorgeous. I had full shadows and bloom effects, and the lighting was amazing. And I was still coming through at 40-60 fps.

THAT is the power of optimization. Bethesda did an absolutely amazing job at making their game look gorgeous even without a video card that would normally handle it. When a company puts in the effort to optimize things well, it can take a game from being mediocre to outstanding.

So keep it up. Keep those load times down so that when I play CK3, I feel like the only bottleneck in my enjoyment of the game is my own ability to make decisions, I'm not ever sitting and waiting on the game.
And Stellaris is great on that too. The loads aren't anywhere near 'instant', but neither can I go grab a soda from the fridge without coming back to my game being fully loaded.
 

MichelsBR

Corporal
5 Badges
Nov 12, 2018
40
38
  • Crusader Kings II
  • Crusader Kings II: The Old Gods
  • Stellaris: Digital Anniversary Edition
  • Crusader Kings III
  • Crusader Kings III: Royal Edition
Thanks, I currently start CK3 and I'm going to take a shower or eat something, on the way back I'll see if it's loaded
 

Vasoli

Sergeant
77 Badges
Mar 21, 2015
68
49
  • Cities in Motion
  • Europa Universalis 4: Emperor
  • Stellaris: Necroids
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Hearts of Iron IV: Expansion Pass
  • Europa Universalis IV: Third Rome
  • Europa Universalis IV: Cradle of Civilization
  • Europa Universalis IV: Mare Nostrum
  • Europa Universalis IV: Rights of Man
  • Stellaris: Synthetic Dawn
  • Stellaris: Humanoids Species Pack
  • Stellaris: Ancient Relics
  • Surviving Mars
  • Imperator: Rome
  • Europa Universalis IV: Common Sense
  • Europa Universalis IV: Cossacks
  • Stellaris: Digital Anniversary Edition
  • Stellaris: Leviathans Story Pack
  • Stellaris: Federations
  • Stellaris: Lithoids
  • Europa Universalis IV
  • Stellaris - Path to Destruction bundle
  • Stellaris: Megacorp
  • Europa Universalis IV: Dharma
  • Stellaris: Distant Stars
  • Imperator: Rome Deluxe Edition
  • Surviving Mars: Digital Deluxe Edition
  • Europa Universalis IV: Rule Britannia
  • Stellaris: Apocalypse
  • Europa Universalis IV: Golden Century
  • Europa Universalis IV: Mandate of Heaven
  • Victoria 2: Heart of Darkness
  • Victoria 2: A House Divided
  • Semper Fi
  • Victoria: Revolutions
  • Europa Universalis IV: Res Publica
  • Magicka
  • Hearts of Iron III
  • For the Motherland
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis III
  • Europa Universalis IV: Art of War
  • Crusader Kings II
  • Crusader Kings II: Horse Lords
  • Crusader Kings III: Royal Edition
  • Cities: Skylines - After Dark
  • Crusader Kings II: Conclave
Nice Dev Diary. Learning how games works is part of appeal of PDX games. I really like that I can see every game detail in text files and adjust them to my liking. So keeping a text version (rather then binary) of database is huge plus in my opinion.

That said, I am having one question in other direction:

I am pretty sure that in older PDX games some text during load time was related to caching. And in this diary I did not see anything related to caching.

So did dev team considered some binary caching of data. Let's say that on the launcher, user can have option to load game from last cached state instead from game files. If he opt to use that option he will have much faster load time. User is more likely to load 2 (or more times) same set of dlcs/mods than to change those selection. And then when game is started, game may in some background task check if something is changed. Game then can write somewhere that information that something is changed and offer the user possibility to load changed version or keep cached version. Maybe even allow user to have multiply cached version of the game (with different dlc/mod selection). I think that this can improve user experience by reducing overage loading time, this would be like having the cake and eating it at the same time.
 

x4077

Lt. General
Sep 16, 2020
1.617
3.450
Huh can’t say I’ve had the same experience with my replays of the legendary edition, I’ve skipped plenty of scenes on redos.
Unfortunately I couldn't skip enough of them to not annoy the bejesus out of me. It became especially annoying having to wait through minor 15-30 second cutscenes over and over again when the last save was right before an encounter that you ended up having to replay multiple times.

The Bank files are because we use a much more sophisticated and better third party audio library called FMOD, our old audio used to be way way too simplistic for what our audio designers and composters wanted.
That makes complete sense, and I can understand it, but what I don't understand is the lack of a user accessible config file somewhere that maps from sound event to sound file in the BANK file. CK2 has it for most/all its sound files in interface/sounds.sfx, CK3 has it for the some of the event backgrounds in common/event_backgrounds/01_event_backgrounds.txt. It's just a shame that there isn't as much control for the user over the sounds as there used to be.

Just some food for future thought - thanks for all the insight and effort providing that insight to us @blackninja9939, I gripe a lot but I do appreciate the work being done.
 

Psieye

Sergeant
64 Badges
Oct 26, 2017
52
78
  • Stellaris: Humanoids Species Pack
  • Crusader Kings III
  • Hearts of Iron IV: Together for Victory
  • Crusader Kings II: Monks and Mystics
  • Europa Universalis IV: Mandate of Heaven
  • Hearts of Iron IV: La Resistance
  • BATTLETECH
  • Surviving Mars
  • Hearts of Iron IV: Death or Dishonor
  • Prison Architect: Psych Ward
  • Age of Wonders III
  • Europa Universalis IV: Cradle of Civilization
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Leviathans Story Pack
  • Stellaris: Apocalypse
  • Europa Universalis IV: Rule Britannia
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Distant Stars
  • Europa Universalis IV: Dharma
  • Surviving Mars: First Colony Edition
  • Crusader Kings II: Holy Fury
  • Europa Universalis IV: Golden Century
  • Prison Architect
  • Stellaris: Lithoids
  • Stellaris: Ancient Relics
  • Europa Universalis IV: Rights of Man
  • Empire of Sin
  • Stellaris: Federations
  • Stellaris: Necroids
  • Europa Universalis 4: Emperor
  • Europa Universalis IV
  • Island Bound
  • Battle for Bosporus
  • Europa Universalis IV: El Dorado
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: Common Sense
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Conclave
  • Stellaris
  • Hearts of Iron IV: Cadet
  • Crusader Kings II: Reapers Due
  • Stellaris: Digital Anniversary Edition
  • Crusader Kings II: Jade Dragon
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Mare Nostrum
  • Stellaris - Path to Destruction bundle
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: The Republic
Here is the dependency graph I captured recently for just our logic
I have dabbled in using various graph visualisation solutions over the years. What did you use to make this visualisation of your dependency graph? Did it convert straight from your object in code to the graph or via an intermediate text dump? And if you are permitted to answer, what class is the graph object in your code?
 

x4077

Lt. General
Sep 16, 2020
1.617
3.450
The reason for this question has actually nothing to do with CK3. When I heard SDL, I remembered SHIFT + Click never working for me in Stellaris, and being instead interpreted as a regular click, which was really frustrating... And I have a suspicion that the reason for this is that SDL-games just don't handle SHIFT + Click properly on Linux.
Having done keyboard/mouse handling code for SDL/SDL2, pretty sure that isn't the fault of SDL, but the fault of the programmers. Mouse operations are not keystrokes, they are handled differently, and unless the programmer is checking for the state of modifier keys in the mouse click event handling code there is no distinction between the two. Even the raw keyboard events generally just send the code for the key pressed, not whether you pressed 'a' or 'A'. Some libraries provide some basic handling for keyboard states for alpha keys ( ie. Shift+A, Ctrl-C, Alt+Q ), some key combinations are translated into different messages based on the OS ( ie. Alt+F4 getting translated into WM_QUIT on Windows ), and others do nothing but provide you the raw data.

Autohotkey has some good documentation about some of the different modes available for dealing with key detection on the Windows side and is a very good read on the subject.
 

blackninja9939

Experienced Programmer - Crusader Kings 3
76 Badges
Aug 28, 2013
2.338
5.134
  • Crusader Kings III
  • Stellaris: Federations
  • Battle for Bosporus
  • Stellaris: Necroids
  • Europa Universalis 4: Emperor
  • Europa Universalis IV
  • Crusader Kings III: Royal Edition
  • Crusader Kings II
  • Crusader Kings II: Holy Fury
  • Imperator: Rome - Magna Graecia
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: The Old Gods
  • Europa Universalis IV: Rights of Man
  • Europa Universalis IV: Cradle of Civilization
  • Stellaris: Synthetic Dawn
  • Surviving Mars
  • BATTLETECH
  • Europa Universalis IV: Mandate of Heaven
  • Crusader Kings II: Monks and Mystics
  • Tyranny: Archon Edition
  • Europa Universalis IV: Rule Britannia
  • Crusader Kings II: Reapers Due
  • Hearts of Iron IV: Colonel
  • Stellaris Sign-up
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Apocalypse
  • Stellaris: Lithoids
  • Stellaris: Distant Stars
  • Europa Universalis IV: Dharma
  • Shadowrun Returns
  • Imperator: Rome Deluxe Edition
  • Prison Architect
  • Imperator: Rome Sign Up
  • Stellaris: Ancient Relics
  • Age of Wonders: Planetfall
  • Crusader Kings II: Conclave
  • Crusader Kings II: The Republic
  • Victoria 2
  • Cities: Skylines
  • Europa Universalis IV: El Dorado
  • Crusader Kings II: Way of Life
  • Stellaris
  • Mount & Blade: Warband
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: Sword of Islam
I have dabbled in using various graph visualisation solutions over the years. What did you use to make this visualisation of your dependency graph? Did it convert straight from your object in code to the graph or via an intermediate text dump? And if you are permitted to answer, what class is the graph object in your code?
I used the “DOT” format as an intermediary text dump and then online tools like GraphVis and a windows explorer plug in to see that text sump visualised.

In our code our directed graph type is our own made class by our engine team, so without copy pasting a lot of code is hard to share anything meaningful of.

It’s core is built off of storing nodes and edges and letting you put data into each of them, combined with a policy that describes the rules for how it works. So for my graph it uses an acyclic policy that causes asserts if you try to make nodes depend on each other through some path as that would not work to load.


So for the nodes I can store things like the function they use to initialise (if any), the state they are currently in, how long they took to load etc. For my uses I didn’t have to store any data on the connecting edges since I could determine all other state and progress via the unique nodes data, but in a different method or if the system becomes more complex that’d be then doable.
 
  • 5
Reactions:

magsthereal

Recruit
11 Badges
Jul 16, 2021
2
0
  • Cities in Motion 2
  • Crusader Kings II
  • Europa Universalis IV
  • Cities: Skylines
  • Mount & Blade: With Fire and Sword
  • Stellaris: Digital Anniversary Edition
  • Age of Wonders III
  • Imperator: Rome Deluxe Edition
  • Imperator: Rome
  • Imperator: Rome - Magna Graecia
  • Crusader Kings III
So there is a difference between debug mode and the debug builds.

Debug builds are what we have internally, it is a different exe with a variety of extra harder errors and asserts as well as having optimisations done by the compiler by and large disabled. That has a lot of things and the lack of compiler optimisations makes all of the loading, and everything really, a lot slower.

Debug mode which is the in game/command line toggle effects things a lot less, the pure exe is still optimised a lot more heavily so its only more minimal overhead since it enables some more logging and other debug paths. Debug mode is what allows the hot reloading of files, we attach file watchers to the files which detect when they are saved and then reload something. In our debug builds we also have debug mode enabled by default since we also want all these debug options on top of the changed exe ones.
So do i understand that correctly, there is an actual difference between using "-debug_mode" and enabeling it ingame via a gui mod?
 

pengoyo

Colonel
66 Badges
Dec 9, 2015
822
2.214
  • Crusader Kings III
  • Crusader Kings II
  • Cities: Skylines
  • Stellaris
  • Imperator: Rome
  • Prison Architect
  • Magicka 2
  • Europa Universalis IV
  • Cities in Motion 2
  • Hearts of Iron IV: Cadet
So do i understand that correctly, there is an actual difference between using "-debug_mode" and enabeling it ingame via a gui mod?
Pretty sure no (though see edit)

Debug mode covers both any mod that enabled it as well as the regular "-debug_mode" version. They are just different ways of enabling it (though a particular mod might add some new custom commands or interfaces).

While debug builds would be a special version of the game only the devs have (i.e. not available to the public) that you can think of as a much more advanced version of debug mode.

edit: see response below, but there might be a different in watched or flagged files depending on if debug_mode is turned on by the mod when loading file in.
 
Last edited:
  • 1
Reactions:

Vasoli

Sergeant
77 Badges
Mar 21, 2015
68
49
  • Cities in Motion
  • Europa Universalis 4: Emperor
  • Stellaris: Necroids
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Hearts of Iron IV: Expansion Pass
  • Europa Universalis IV: Third Rome
  • Europa Universalis IV: Cradle of Civilization
  • Europa Universalis IV: Mare Nostrum
  • Europa Universalis IV: Rights of Man
  • Stellaris: Synthetic Dawn
  • Stellaris: Humanoids Species Pack
  • Stellaris: Ancient Relics
  • Surviving Mars
  • Imperator: Rome
  • Europa Universalis IV: Common Sense
  • Europa Universalis IV: Cossacks
  • Stellaris: Digital Anniversary Edition
  • Stellaris: Leviathans Story Pack
  • Stellaris: Federations
  • Stellaris: Lithoids
  • Europa Universalis IV
  • Stellaris - Path to Destruction bundle
  • Stellaris: Megacorp
  • Europa Universalis IV: Dharma
  • Stellaris: Distant Stars
  • Imperator: Rome Deluxe Edition
  • Surviving Mars: Digital Deluxe Edition
  • Europa Universalis IV: Rule Britannia
  • Stellaris: Apocalypse
  • Europa Universalis IV: Golden Century
  • Europa Universalis IV: Mandate of Heaven
  • Victoria 2: Heart of Darkness
  • Victoria 2: A House Divided
  • Semper Fi
  • Victoria: Revolutions
  • Europa Universalis IV: Res Publica
  • Magicka
  • Hearts of Iron III
  • For the Motherland
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis III
  • Europa Universalis IV: Art of War
  • Crusader Kings II
  • Crusader Kings II: Horse Lords
  • Crusader Kings III: Royal Edition
  • Cities: Skylines - After Dark
  • Crusader Kings II: Conclave
So do i understand that correctly, there is an actual difference between using "-debug_mode" and enabeling it ingame via a gui mod?
Pretty sure no.

Debug mode covers both any mod that enabled it as well as the regular "-debug_mode" version. They are just different ways of enabling it (though a particular mod might add some new custom commands or interfaces).

While debug builds would be a special version of the game only the devs have (i.e. not available to the public) that you can think of as a much more advanced version of debug mode.
You really do not want to use debug builds while playing the game, or using any software for that matter. Debug builds might have some "additional" feature compared to release builds ( marked in c++ code with something like #ifdef DEBUG) but usually there aren't many of those. That kind of "features" are usually there to overcome some limitation of your environment, when you want some special behavior during debug. But during debug you usually want your application to behave as closer as possible to how it would actually behave.

What debug build is mostly adding additional information so execution of certain part of the code can be recognized and displayed in debugger.

For example somewhere in code you can have something like this:
Code:
auto a = 1;
auto b = a;//and after this a not used anymore in rest of the code
auto c = b;//and after this b not used anymore in rest of the code
auto d = c;//and after this c not used anymore in rest of the code
auto e = d;//and after this d not used anymore in rest of the code
auto f = e;//and after this e not used anymore in rest of the code
// now do something with f
// you will not have this example in real code, I am just showing it as example to explain what can be optimised
In release build, compiler can say, I do not need a, b, c, d, e variables anywhere, so I will replace all of this with auto f = 1; This is much faster and used much less memory. But in debug build that replacement is never done. In debug mode, you want to see when variable a get value of 1, then variable b etc. You want to know when any of those lines are executed.

Of course this is absurd example, you will never write this kind of code. But there are much more subtle optimization, where code is reduced and speed up during building release. This is just extreme example to show the point.

So if you are using software, and you are not trying to figure out why it is not working properly, you always want to use release build, and not debug build.

And about differences between "-debug_mode" and enabeling it ingame via a gui mod, I would say that there are some additional differences. File watchers are for sure attached on load time, so you will not probably have that feature with gui mod. And maybe some logging is turned on as well. I cannot come up with more ideas what "-debug_mode" can have more then gui mod. But all of that, if any additional feature exists, must be related to that flag being present during load time. And all of these are not features related to debug build, but to debug mod in release build.
 
  • 2
  • 1Like
Reactions: