• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

EU4 - Development Diary - 9th of April 2019

Bonjour everyone! It is I, Le French Paradox and I will be your host for this Dev Diary.

Today we will talk about Tech Debt. As @DDRJake has been saying, tech debt has been an important focus for the EU4 team since the release of 1.28.3, the latest support patch we did for the Spain update. While our designers are thinking very hard about the yet-to-be-announced big expansion that is to come, we programmers have been busy tidying up the EU4 codebase.

tidying up.jpg

Microsoft should team up with Netflix, I would watch the hell out of this show

Jokes aside, we keep talking about Tech Debt but realize it might be a bit confusing, especially for our non-programming crowd.

While EU4 was announced at GamesCon 2012, the actual development started a year before, which makes the code 8 years old. And of course, the team at the time didn't reinvent everything, bits were pulled from EU3, some of which are still in use today. I'll let you speculate which is which in the comments.

As I'm sure you all know, technology is a rapidly evolving thing, and software engineering is no exception. In 10 years the development practices have evolved, both because the industry keeps learning and inventing new ways to solve problems, and because the ecosystem (the hardware and software that runs our games) also changes.

10 years ago, SSDs were not that widespread, 1080p was pretty new, DirectX 10 was all the rage in previews and most people were so hostile to Windows Vista that they kept running XP in 32 bits.
That is not to say that EU4 was using only stuff from the late 2000s or early 2010s until now. Over time our engineers have done their best to keep the game up to date, but some stuff inevitably slips through the cracks. It's the accumulation of all this aging stuff that we call tech debt(*).

Here's a few examples of those things that we have tackled so far.

64 bits
As the BattlePope already explained in a past DD, 32 bits was a reasonable choice when EU4 development started, but today 64 bits in the norm.
With that done EU4 will keep running with the next update of MacOS (which removes 32 bit compatibility altogether) and also be able to use more RAM. While there are no immediate plans to increase the memory usage of the game significantly, mods that add a lot of provinces and tags should be able to break the previous ceiling.

Rendering
It is no secret that our games do not use cutting edge graphics. True to its board game heritage, Europa Universalis is mostly about showing a world map with some cool dudes (and elephants) fighting on top of it. Yet if you take a look at the Imperator preview, you will see that our graphics experts have learnt a trick or two since EU4's release.

2xnk41.jpg


While Boromir is (was?) right, we still managed to replicate some of the improvements they have made. The main one is that the colorizing of the provinces on the map is now done mostly on the GPU, while in the past it used to consume precious CPU cycles to display whatever horrible experiment in bordergore your game was about.

capture.PNG

Let's play 7 differences, can you spot what changed from the current release in this beta screenshot?

Crash reporter
Our game never crashes, it just tactically exits. Yet when it happens, we want to know all about it. It helps our QA to reproduce the issue and our programmers to isolate the piece of code responsible and fix it.
Since EU4's release our engine team has made a new and improved crash reporter that brings us more data and help find problems faster. First of all, it now works on Linux and MacOS, meaning we will be able to investigate issues on those platforms much easier. Secondly, it allows us to add some metadata to the dump, such as the current year, the list of enabled mods or how many 6/6/6 heirs died to a hunting accident, helping us understand what triggered the issue.

Startup time
Improving the loading time of the game is something we always want to do. While we can't really apply the best fix to the issue (discontinue Windows support, the game just starts so much faster on Linux and MacOS, trust me), we managed to find some things we could do.
The most notable one was the upgrade of PhysFS, a 3rd party software that a lot of videogames use to load all resources (files on disk, mods, DLCs...). While still far from Unix performance, it should shave off a couple seconds from the startup time on Windows.

capture.PNG

Experienced Byzantium players recommend playing on Linux or Mac due to the very fast restart time

General performance
The speed of the game is, of course, always a concern. Every night, we run a couple games on benchmark machines and always make sure that the averages are below a certain value (80ms per in-game day is our current high threshold).
While you may think that performance would improve over time (due to new hardware for example), in practice the average usually go up during the development phase of a new expansion, and is then worked upon to bring it back to acceptable values.
Contrary to popular thought, adding new features in the game is not the only (or even main) source of performance regression. In fact, one big factor is simply the addition of tags and provinces.
Consider this: for every two countries in the game, a bunch of stuff needs to be computed each in-game day (relations, AI attitude...). For every two provinces in the game, the game has to know the way from A to B (what we call pathfinding). This is known as a quadratic problem. For those who don't favor math, it means that for X provinces, the problem complexity is X squared. Doubling the number of provinces does not double the number of computations needed, it quadruples it. Over the years, part of our job has been to allow EU4 to grow from around 2000 provinces to more than 4000 provinces.
Keep in mind though that in software engineering an optimization that makes something 0.1% faster is considered pretty good, 1% excellent and 10% probably means you introduced a serious bug (or fixed a very serious one). How do we manage to improve anything with those numbers? Simple: all those add up in the end.

That's all for today! Should you like to know more meaty details of the tech stuff, you can check-out my blog where I get into more practical details about development practices.

Next week we should return to a more classic Dev Diary by our designers, unless you can help me convince Jake to revamp the wine trade good to include grape kinds and soils and finally settle which of the Burgundian or Bordelaise tradition of winemaking is the best.

(*) It's a bit more complicated than that but should be enough for the sake of this diary
 
Last edited:
Contrary to popular thought, adding new features in the game is not the only (or even main) source of performance regression.
In fact, one big factor is simply the addition of tags and provinces.
Yeah, but I've doubts, that the addition of even more provinces is such a high request ...

10 years ago, there was a game called "Empire: Total War" in which you've played in the napoleonic era, but for some "reasons" the devs had the "good" idea, that something like the motherland of france (or spain) had to consist just 1 province. That's a case in which you have to add more provinces, but if I'm observing the ongoing EU-IV-development-process or the rather absurdly large map of the upcoming "Imperator: Rome" then I'm thinking myself: Seriously guys and girls, it's enough since at this point, you're just crippling the game-performance and you're forcing the player to deal with even more of the "bad" aka repetitive micromanagement since the addition of even more provinces doesn't change the actual game-mechanics since you're still doing the same stuff as before, it's just ... well ... more.
 
Wait? So I've always been embarrassed for using my Mac to game with, but it's actually better??

Not better, in-game performances are usually pretty inferior on Mac, sadly. But startup time... god... I can launch and close the game three times on my Macbook before a gaming PC will be able to enter the main menu. So the strategy is: use Mac or Linux to kickstart your Byzantium and then once you consolidated your starting point move on Windows ;)
 
If i'm not mistaken,It's for reload the game to prevent some issues with game initialisation,especially for mod debugging.
If I recall, it also acted as a fix to an ironman exploit whereby quitting back and forth to menu from game start-up would allow you to change the difficulty/lucky nations/etc that you can't normally do. Presumably, restarting from the desktop clears everything out and prevents any weirdness from a previously loaded save (I know similar things can happen if you shuffle back and forth between CKII bookmarks).
 
Not better, in-game performances are usually pretty inferior on Mac, sadly. But startup time... god... I can launch and close the game three times on my Macbook before a gaming PC will be able to enter the main menu. So the strategy is: use Mac or Linux to kickstart your Byzantium and then once you consolidated your starting point move on Windows ;)
And the Linux performance are good compared to Windows or this is less good than Windows?
Thanks for any reply about this.
 
If I recall, it also acted as a fix to an ironman exploit whereby quitting back and forth to menu from game start-up would allow you to change the difficulty/lucky nations/etc that you can't normally do. Presumably, restarting from the desktop clears everything out and prevents any weirdness from a previously loaded save (I know similar things can happen if you shuffle back and forth between CKII bookmarks).
Ok i didn't know that.Thanks for the info.
 
Not better, in-game performances are usually pretty inferior on Mac, sadly. But startup time... god... I can launch and close the game three times on my Macbook before a gaming PC will be able to enter the main menu. So the strategy is: use Mac or Linux to kickstart your Byzantium and then once you consolidated your starting point move on Windows ;)
Do you know why it is that when mods are loaded the first time it seems to take forever but it the same mod list is loaded a second time it runs much faster? Or is that just what it feels like to me?
 
And the Linux performance are good compared to Windows or this is less good than Windows?
Thanks for any reply about this.

As far as I know performances on Linux are on par with Mac ones.

Do you know why it is that when mods are loaded the first time it seems to take forever but it the same mod list is loaded a second time it runs much faster? Or is that just what it feels like to me?

There's a lot of caching work when you load a new mod (or when you load the game for the first time after an update. So yes, it's much faster after the first time, you are right ;)
 
Differences on screenshot are:

-Colored borders on HRE map mode.
The country color visible on HRE map mode.

-Lorraine has orange stripes, maybe this means a HRE member is a vassal outside the empire? {imperial liberation cb maybe}

- Barrois wasn't part of the HRE before I think. . Correct me if I'm wrong.

-More Free Cities then before.

-also I can't recall the red color that the Emperor has, but I hardly use the HRE map mode so I could be wrong.


Please add to the list and correct me if you know more.
 
Not better, in-game performances are usually pretty inferior on Mac, sadly. But startup time... god... I can launch and close the game three times on my Macbook before a gaming PC will be able to enter the main menu. So the strategy is: use Mac or Linux to kickstart your Byzantium and then once you consolidated your starting point move on Windows ;)

Thank you!

I do need a new laptop too... ;)
 
Whenever I go to load into a game and then have to exit to main menu the game completely restarts - as in it exits to desktop and then reloads.
Will you guys fix the "Return to Menu" problem? It's quite annoying that we have to restart the whole game just to get back to the menu. ((No offense, just asking
@MatRopert will next patch fix thing with "press back from nation select screen" and game restarts and loads anew? It is infuriating.
The "return to menu" is a legacy from EU3 in which going back to the menu after you start playing will result in a different checksum.
In EU4 it was quickly addressed by the "exit and restart to menu".
We have been wanting to fix it entirely for quite some time now but I can't make any promises as the cost is expected to be high and we have to prioritize it along many other things we could also look at (and, of course, the gameplay content of the next big patch).
Then again we also thought that we would never manage to make hotjoin work and in the end we did, so who knows?
 
Last edited:
Hmm... Did the hatching on different provinces line up before? (like HRE provinces not under HRE control). Apart from that I'm not sure I can spot any difference.
You're getting warm but not exactly there yet.
 
The "return to menu" is a legacy from EU3 in which going back to the menu after you start playing will result in a different checksum.
In EU4 it was quickly addressed by the "restart to menu".
We have been wanting to fix it entirely for quite some time now but I can't make any promises as the cost is expected to be high and we have to prioritize it along many other things we could also look at (and, of course, the gameplay content of the next big patch).
Then again we also thought that we would never manage to make hotjoin work and in the end we did, so who knows?

In a way I like that it takes a lot to reload. It makes cheating more costly for me: "Yes, I could restart until I get that perfect start, but this above average one is fine" or "My army was stack wiped by bad dice rolls and I'm raging, but man, do I want to wait 20 mins to try again? Nah..."

PS: I'm definitely exaggerating
 
Improving the loading time of the game is something we always want to do.

Honestly, I give 0 coalition wars about loading time. It's only another reason to go and make some english tea before "colonizing" another barbarian. What I do care is actual problems: like truce timer not showing up properly, or slow down in late game wars, AI acting like dummie: not building forts whatnot. You know, actual problems.
 
Good diary that also is an eye opener on the other Paradox titles.

Think you might have persuaded more people to try Linux btw