• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

EU4 - Development Diary - 9th of April 2019

Bonjour everyone! It is I, Le French Paradox and I will be your host for this Dev Diary.

Today we will talk about Tech Debt. As @DDRJake has been saying, tech debt has been an important focus for the EU4 team since the release of 1.28.3, the latest support patch we did for the Spain update. While our designers are thinking very hard about the yet-to-be-announced big expansion that is to come, we programmers have been busy tidying up the EU4 codebase.

tidying up.jpg

Microsoft should team up with Netflix, I would watch the hell out of this show

Jokes aside, we keep talking about Tech Debt but realize it might be a bit confusing, especially for our non-programming crowd.

While EU4 was announced at GamesCon 2012, the actual development started a year before, which makes the code 8 years old. And of course, the team at the time didn't reinvent everything, bits were pulled from EU3, some of which are still in use today. I'll let you speculate which is which in the comments.

As I'm sure you all know, technology is a rapidly evolving thing, and software engineering is no exception. In 10 years the development practices have evolved, both because the industry keeps learning and inventing new ways to solve problems, and because the ecosystem (the hardware and software that runs our games) also changes.

10 years ago, SSDs were not that widespread, 1080p was pretty new, DirectX 10 was all the rage in previews and most people were so hostile to Windows Vista that they kept running XP in 32 bits.
That is not to say that EU4 was using only stuff from the late 2000s or early 2010s until now. Over time our engineers have done their best to keep the game up to date, but some stuff inevitably slips through the cracks. It's the accumulation of all this aging stuff that we call tech debt(*).

Here's a few examples of those things that we have tackled so far.

64 bits
As the BattlePope already explained in a past DD, 32 bits was a reasonable choice when EU4 development started, but today 64 bits in the norm.
With that done EU4 will keep running with the next update of MacOS (which removes 32 bit compatibility altogether) and also be able to use more RAM. While there are no immediate plans to increase the memory usage of the game significantly, mods that add a lot of provinces and tags should be able to break the previous ceiling.

Rendering
It is no secret that our games do not use cutting edge graphics. True to its board game heritage, Europa Universalis is mostly about showing a world map with some cool dudes (and elephants) fighting on top of it. Yet if you take a look at the Imperator preview, you will see that our graphics experts have learnt a trick or two since EU4's release.

2xnk41.jpg


While Boromir is (was?) right, we still managed to replicate some of the improvements they have made. The main one is that the colorizing of the provinces on the map is now done mostly on the GPU, while in the past it used to consume precious CPU cycles to display whatever horrible experiment in bordergore your game was about.

capture.PNG

Let's play 7 differences, can you spot what changed from the current release in this beta screenshot?

Crash reporter
Our game never crashes, it just tactically exits. Yet when it happens, we want to know all about it. It helps our QA to reproduce the issue and our programmers to isolate the piece of code responsible and fix it.
Since EU4's release our engine team has made a new and improved crash reporter that brings us more data and help find problems faster. First of all, it now works on Linux and MacOS, meaning we will be able to investigate issues on those platforms much easier. Secondly, it allows us to add some metadata to the dump, such as the current year, the list of enabled mods or how many 6/6/6 heirs died to a hunting accident, helping us understand what triggered the issue.

Startup time
Improving the loading time of the game is something we always want to do. While we can't really apply the best fix to the issue (discontinue Windows support, the game just starts so much faster on Linux and MacOS, trust me), we managed to find some things we could do.
The most notable one was the upgrade of PhysFS, a 3rd party software that a lot of videogames use to load all resources (files on disk, mods, DLCs...). While still far from Unix performance, it should shave off a couple seconds from the startup time on Windows.

capture.PNG

Experienced Byzantium players recommend playing on Linux or Mac due to the very fast restart time

General performance
The speed of the game is, of course, always a concern. Every night, we run a couple games on benchmark machines and always make sure that the averages are below a certain value (80ms per in-game day is our current high threshold).
While you may think that performance would improve over time (due to new hardware for example), in practice the average usually go up during the development phase of a new expansion, and is then worked upon to bring it back to acceptable values.
Contrary to popular thought, adding new features in the game is not the only (or even main) source of performance regression. In fact, one big factor is simply the addition of tags and provinces.
Consider this: for every two countries in the game, a bunch of stuff needs to be computed each in-game day (relations, AI attitude...). For every two provinces in the game, the game has to know the way from A to B (what we call pathfinding). This is known as a quadratic problem. For those who don't favor math, it means that for X provinces, the problem complexity is X squared. Doubling the number of provinces does not double the number of computations needed, it quadruples it. Over the years, part of our job has been to allow EU4 to grow from around 2000 provinces to more than 4000 provinces.
Keep in mind though that in software engineering an optimization that makes something 0.1% faster is considered pretty good, 1% excellent and 10% probably means you introduced a serious bug (or fixed a very serious one). How do we manage to improve anything with those numbers? Simple: all those add up in the end.

That's all for today! Should you like to know more meaty details of the tech stuff, you can check-out my blog where I get into more practical details about development practices.

Next week we should return to a more classic Dev Diary by our designers, unless you can help me convince Jake to revamp the wine trade good to include grape kinds and soils and finally settle which of the Burgundian or Bordelaise tradition of winemaking is the best.

(*) It's a bit more complicated than that but should be enough for the sake of this diary
 
Last edited:
@MatRopert Will these GPU changes fix the odd coloring issues that are seen every now and then? Example: Savoy's pink colour tends to be very grainy, not 100% the same colour.

Or province coloring not covering all of a border province.
I'm not sure... A screenshot in a post on the bug report board (with save and possibly dxdiag output) would certainly help us give a proper answer ;)
 
I don't recall a problem like this. Did you make a bug report?
Today's crash dumps are already pretty telling on what went wrong.
The tool itself will help for those edge cases where your game crashes but we can't find a reproducible way to trigger it.
I don't remember if I made a bug report as I have this problem for years. However I send crash reports frequently. Anyway I'm pretty sceptical on reporting non-reproducible bug which appears after playing several hours with M&T mod.
 
I might have been a bit confusing here.
When I said "play 7 differences" it's a reference to a french kid game that may or may not translate well.
In this particular image there should be mostly one difference, but maybe you'll spot something I didn't :)
Ah ok, in my native language its "find five differences" haha. Jamais entendu parler de "trouve sept differences" par contre haha
Still I can see more than one difference in the screenshots, perhaps you haven't been informed of all the changes lol
 
I take you're playing on Windows? NTFS (the way Windows arrange your files on your disk) is notably bad at accessing a lot of small files (such as our history files).
This is the main reason why Linux and MacOS starts much faster.
An obvious solution would be to put all core data in one big zip file but it's trickier than it sounds.
Yes, I'm running Windows 7 ultimate 64bit.
Occasionally the game starts faster if I've recently been playing, but usually it takes forever to start the game, and it's specifically the "loading history files" part that takes time.
Anyway, what you're saying is that there isn't really anything I can do except for changing my OS?
Thanks for taking your time to answer my questions.
 
Previous Dev Diaries only started when they had something to show. This time they're starting from more or less the ground up before any concrete decisions have been made, so it makes sense there isn't any specific info yet, just general intents.


This is true. Personally, I thought that when they talked about an expansion almost a year in the making, that it would be..... a year in the making. So far the only evidence of new content that have revealed are a piece of paper with a Burgundy mission tree and ideas on what they want to do, and not do, with the map update. If it takes them longer than expected and we get a new expansion which adds more detail and ways to play, I am perfectly fine with that. But if it ends up being more insta-click buff buttons, and dev diaries solely based on that I will be extremely disappointed.
 
This dev diary convinced me to use Ubuntu.

I only run the game on Ubuntu. Once, as an experiment, I crash/loaded 500 times in response to the "heir falls ill" event to see if the outcome is predetermined. It took me 12 minutes total (not including the time to tally and do maths).
 
Last edited:
I only run the game on Ubuntu. Once, as an experiment, I crash/loaded 500 times in response to the "heir falls ill" event to see if the outcome is predetermined. It took me 12 minutes total (not including the time to tally and do maths).

But do you feel the performance ingame changes? Or is it just launching?
 
But do you feel the performance ingame changes? Or is it just launching?

I used to play on a much more powerful Windows gaming PC, but now I prefer to play on my refurbished Ubuntu laptop. The downside is not being able to exploit the "click the mission thousands of time while the game is busy saving to develop the capitol way too high" bug back when that was a thing people were doing, because the save tick passes (basically) just as quick as the monthly ticks.
 
Will you do some modification to the code to support non-english language rule in "automated" part (for instance, transforming in french "de le" into "du" or "le+vowel" into "l'+vowel", support adjective declinaisons in german, russian, french etc)? Or is localisation not a priority or in your scope at all?
 
Just because we are talking so much about performance and amounts of ms per calculated day: What would you consider the best bang for buck EU4-processor at the moment - and does it really make a huge difference on game perfomance?
 
the game just starts so much faster on Linux
Can confirm.

Contrary to popular thought, adding new features in the game is not the only (or even main) source of performance regression. In fact, one big factor is simply the addition of tags and provinces.
I thought it was common knowledge that more tags and provinces caused slowdown, though perhaps popular thought has changed. I know that it at least was common knowledge back in the days of EU3 and early days of EU4.

Not better, in-game performances are usually pretty inferior on Mac, sadly. But startup time... god... I can launch and close the game three times on my Macbook before a gaming PC will be able to enter the main menu. So the strategy is: use Mac or Linux to kickstart your Byzantium and then once you consolidated your starting point move on Windows ;)
In my experience the game has good performance on Linux, though I'm using a laptop and has for years, so perhaps I just am not aware of how fast it can be on a gaming rig as opposed to a gaming laptop.
 
When I said "play 7 differences" it's a reference to a french kid game that may or may not translate well.
My experience as a native English speaker is that the game in question is usually called something like "spot the difference(s)", so if someone mentions an explicit number we'll tend to take it literally...
 
While I am on a bad laptop
- Intel Core i3 with 2.00GHz
- 4GB of installed memory (RAM)
- 64-Bit operating system

I can run the game. The start up time is usually about 5 minutes and while it's a pain, I have a bigger problem.
When I click on single player and then go back to the menu, the whole game closes
(Message says Graphics Card Failed) and restarts (another 4-5 minutes). I assume this is just me but I don't know.
 
Yeah, but I've doubts, that the addition of even more provinces is such a high request ...

10 years ago, there was a game called "Empire: Total War" in which you've played in the napoleonic era, but for some "reasons" the devs had the "good" idea, that something like the motherland of france (or spain) had to consist just 1 province. That's a case in which you have to add more provinces, but if I'm observing the ongoing EU-IV-development-process or the rather absurdly large map of the upcoming "Imperator: Rome" then I'm thinking myself: Seriously guys and girls, it's enough since at this point, you're just crippling the game-performance and you're forcing the player to deal with even more of the "bad" aka repetitive micromanagement since the addition of even more provinces doesn't change the actual game-mechanics since you're still doing the same stuff as before, it's just ... well ... more.

Since Golden Century came out, I've been here every Tuesday hoping today is the day they will adress the criticism with the 1.28 map changes and adding one or two more provinces to Portugal and Central Castille.
Everytime i've left dissapointed. But i still have hope.
My computer is by no means fast, but i'm always in favour of more provincial detail. I'm not asking for a Meiou and Taxes level of detail, but for me its unnaceptable that very strategically, geographically and politically relevant provinces are still not present.

More provinces might make world conquering and playing very wide more tedious, but its tedious anyway, at least it makes playing tall less tedious.
I think i speak for every non-wide player that yes, province density is (almost) always desirable (Ibiza was pointless) and definitely in demand.
 
Last edited:
While I am on a bad laptop
- Intel Core i3 with 2.00GHz
- 4GB of installed memory (RAM)
- 64-Bit operating system
64-bit OS + 4GB of RAM is... not going to be a pleasant platform to run modern software on, to be honest. (Especially if the OS is Windows.)
I can run the game. The start up time is usually about 5 minutes and while it's a pain, I have a bigger problem.
When I click on single player and then go back to the menu, the whole game closes and restarts
It's supposed to (as has been pointed out on every one of the past seven pages, as far as I can tell :) ). I don't know why you get an error message relating to your graphics driver when it does, though.