• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

HoI4 Dev Diary - A Tech Lead's Life

Hello everyone, it's your favourite (and only) tech lead, The French Paradox!

In this diary I'll tell you more about our tech, give some insights into the programmer's job, what we've been up to for the past year (and some change) and offer you a chance to ask any question you may have about tech.

Saying old platforms goodbye

First, with the upcoming 1.11 patch we will be retiring some old Linux and Mac platforms that aren't maintained by their own developers anymore.
This means that starting next update the minimum OS version to run HoI will be Ubuntu 18.04 (on Linux) and macOS 10.11 (El Capitan) on Mac.
There shouldn't be much to worry about though, as the previous versions are older than HoI itself at this point and our telemetry shows almost all of you have upgraded long ago.
For those on Windows who look anxiously at their aging Windows 7, keep calm and carry on, we have no plans to drop support for it (although as a friendly tech person I'd still suggest you consider upgrading ;)).

Why do we make those decisions you may wonder? Two reasons: one maintaining old tech comes with a cost that we would rather spend on improving the game for 99% of the users, and two sometimes it blocks the adoption of new tech that would make our programmers job easier and more efficient.
Without going too much into details, the general idea is that we can only use tech that is supported on all platforms, so basically we're only as modern as the oldest thing we support. If you take the case of Linux for example, the previous Ubuntu release was from 2016, meaning HoI could not use anything released since then (and probably a bit before, as Ubuntu LTS releases usually don't ship with the latest shiny goodies). Some time we can work our way around it, but not always.
Speaking of new tech...

Renderers.PNG


DirectX 11

That's right, the next HoI4 release will join Stellaris and the other more recent PDS games by adding DirectX 11 support.
DirectX 9 will still be the default for now, but you will be able to select another renderer in the launcher settings.

There shouldn't be any visual difference between DX9 and DX11 (and OpenGL for that matter), we have been working on making sure the experience will be the same.
The game should load a bit faster however, especially if you use a lot of mods that bring extra textures and models.
Working remotely for more than a year, we also noticed less issues playing with remote desktop and the like, which might not be a big deal for most of you but was quite appreciable for our devs during development.
Finally it did help us tweaking & fixing the new railways & trains graphics (as most graphics debugging tools have dropped support for DX9).


Performance

We know that performance is always at the heart of every discussion about our games and as a Tech Lead I have been keeping a close eye on it.
So far it looks like our release candidate is roughly on par with the current 1.10 patch, performance wise.

Why not better, I can already hear you typing? The main reason is that the game on 1.11 is a different beast 1.10 (which itself is quite different from the original 1.0 release).
Namely, the supply simulation is now much more deep and complex, and needs to account for all those railways across the world and the fact that now allies can supply each other.
While I am writing those words, our compatibility lab is working hard at running the game on various hardware configurations to give us a more complete picture and perhaps a few suggestions for an updated recommended configuration, as the one you can see on Steam right now dates back from the original release.


20211103_120703.jpg

Toying with a 11th generation i9 and a RTX 3090. Will it take off?

On the topic of measuring performance, the Barbarossa update will include an in-game profiler that you can use to measure how your machine performs.
You should be able to spawn it from the console using the magic line imgui show profiler.

For example here is the current release candidate on my home i7-10700 with a RTX 2080 SUPER:

hoi4_1936_RC.PNG

1936 fresh start, speed 5, Direct 11, vsync off

hoi4_1943_RC.PNG

1943 test save, speed 5, Direct 11, vsync off

As you can see this offers a few metrics that can be collected by toggling the Enable/Disable Collection button at the top.
A few things you can get from it:
  • Render time: the average time to render one frame over the last second
  • Render time excluding present: same as previous but excludes the time spent waiting on the GPU to actually present the frame on screen. The difference is usually due to vsync.
  • Frames per second: the FPS count, you know this one
  • Ticks per second: how many in-game hours were simulated over the course of the last second
  • Last tick: the time it took to simulate the last hour or new day/week/month
  • Last 24 ticks average: same as previous but averaged from the last 24 (or less depending on how much time was spent collecting data)
Next you have a nice graph representation of where (in which system) that time was spent. We use it to quickly eyeball a performance report, especially when external programmer tools are not available (for example when it happens on a non-dev machine).

Final tip: if the profiler doesn't show up with the magic console command, try turning it off and on again using imgui off then imgui on.


A personal pet peeve

Programmers at PDS are rarely just that. Most of us get involved in the game development by offering insights on design, balance or content, and I am no exception.
So now is the time to talk about French Communism.

If you remember the French focus tree from La Résistance, if you go communist you get this guy:

Momo.PNG


Maurice Thorez was historically the leader of the Parti Communiste Français (PCF).
If you walk through some cities in France you can even find streets bearing his name still today.
But he had one characteristic trait: he was a die-hard Stalinist who followed the Moscow line until his death in 1964. He had a city and an institute named after him in the USSR.
On the day the Molotov-Ribbentrop Pact was signed, he is in vacation in the Alps and the PCF changes its stance on Germany and Poland overnight following Moscow's instructions without asking him. When he comes back, he just shrugs and goes along with it.

With that in mind, I felt that it was weird you would keep him as a leader if you do not go for the "Loyalty To Moscow" sub-tree.
I did some research in French sources and found a better candidate to replace him. Behold:

1635946239203.png


René Nicod was a member of the PCF in 1939, but when the party decided to realign itself following the Pact, he and a couple other MPs quit the party and founded the Union Populaire Française (UPF) or French Popular Union in English. They denounced the Pact and supported the government's decision to stand with Poland.
In Barbarossa if you own La Résistance and go either Anti-Fascist Coalition or Loyalty to the cause you will get a new party and leader:

Tree.PNG
Tree2.PNG


Each branch will give a different flavour of Nicod inspired by his published opinions pieces during the Great War (he both supported in the French Army and lost his hand in the trenches but also denounced the war on ideological grounds and hoped the French and German workers would unite in a common cause).

Finally, it turned out to be a bit difficult to find a good portrait for our artists. My initial research in the French National Assembly online database only yielded a low-res blurry picture.
But luckily a small French shop decided to sell its stock of old photos on eBay and his mayor portrait was one of the articles.
We could have stopped there and used the ad's preview picture as a source, but of course we had to buy and ship to Sweden.

So here he is, looking over the HOI corner in our Stockholm office!


IMG_20211102_203447.jpg

IMG_20211102_203534.jpg
And with that, I'll happily take your questions :cool:

PS: a few back we talked about Modding Changes. @Aurelien Delay looked at your feedback and has more insights to share that should help you convert your mod to the new character system.
 
  • 130Like
  • 26Love
  • 12
  • 4
  • 3
Reactions:
Sometimes I ask myself whether the people complaining about the alleged single core usage do a field test and deactivate the other cores on purpose and run the game really on one core.
 
Have you considered adding a soft cap on units to prevent that late game unit spam?

There is already a soft cap - manpower and equipment. Anything other is a hard cap.

And hard caps throw balance and game flow out the window. You'd need an unreasonable amount of work hours of rebalancing and reworking to make it work.

"Just put a hard cap on the ai" is asking for trouble. It is handicapped enough as it is.
 
  • 4
  • 2Like
  • 2
Reactions:
It takes an hour for a modder to add in division caps. It doesnt affect balance at all. Everyone whos ever played EAI mod knows this.

The actual scripting is likely to be fairly quick, but the testing, balancing and tweaking much more involved. Just mentioning this - when I was modding (and will hopefully again soon, albeit at a reduced rate) the scripting was the easy bit - it's working out what to script, and then getting the balance right, that really mattered (for things like this of course - flavour events, new leaders and some other less balance-affecting bits and bobs are just scripting and easy :) )
 
  • 4
  • 1Like
Reactions:
It takes an hour for a modder to add in division caps. It doesnt affect balance at all. Everyone whos ever played EAI mod knows this.

A nation not having divisions above a certain level does not affect balance? :rolleyes:

Did not mean that putting a cap in would take long. Meant that making it so that the gameplay does not suffer will. Modders do not have to bother with game health, or any larger considerations.

But right, devs lazy, wont put one variable in.
 
  • 6Like
Reactions:
@MatRopert , the U.S. flag in your office is being displayed incorrectly! The stars should be on the left, not the right. Here's what I mean:
IMG_20211102_203534_1.jpg
If you would kindly, sir, please flip your U.S. flag on the horizontal, this American will pretend this never happened. In the meantime,
angry clint.png
 
Last edited:
  • 3
  • 1Like
Reactions:
I'm amused that you said that a portrait for René Nicod was difficult to find because the same photo that you used is already in his wikipedia page.
It is now indeed. It wasn't always the case. They used to have the blurry one from the national assembly archives. It took that shop to put a scan on eBay for it to be pulled on Wikipedia.
 
  • 4
  • 4
Reactions:
I agree at 100% with you, whe should have an event to remplace Laval, it always look weird to me to fight against nazis with him leading france while knowing his real history.
When the game's developed enough to get into post-war stuff, I'd like events to deal with folks like him... especially Quisling.
 
There is already a soft cap - manpower and equipment. Anything other is a hard cap.

And hard caps throw balance and game flow out the window. You'd need an unreasonable amount of work hours of rebalancing and reworking to make it work.

"Just put a hard cap on the ai" is asking for trouble. It is handicapped enough as it is.
Manpower and equipment are a soft cap but it's not enough, and many countries produce many more units than they should, creating lag and making the game unplayable for many. I don't know how you doing recognize that this is a problem, or you do it but don't want to apply the most direct solution.

A hard cap means that, when the threshold is met, the game blocks you from producing more units. And that's not what I asked for. I would put debuffs on countries that cross the threshold, but not block them from doing so. And I didn't say "cap the AI", I would cap players too.

Kaiserreich has a soft cap using owned states and factories, among other things, and it works. It's not too harsh but reduces spam. And it's optional: if you don't like it, it can be disabled in game rules. Adding an optional feature can't harm anyone and solves problems for those who have them. Balancing it needs work but it's nothing Paradox can't do.
 
  • 4
Reactions:
@MatRopert , the U.S. flag in your office is being displayed incorrectly! The stars should be on the left, not the right. Here's what I mean:
If you would kindly, sir, please flip your U.S. flag on the horizontal, this American will pretend this never happened. In the meantime,
I recommend reading this:
 
  • 2
Reactions:
Are you considering switching from a lot of text files to SQL DB?
I believe it could make the game loading faster.
Ins't NoSQL faster? Might be better for the modders, too - just have to update JSON files.


Soo what this dev diary should be telling us is the game wont die after the 1940s
Considering what he said on the dev diary:


So far it looks like our release candidate is roughly on par with the current 1.10 patch, performance wise.

Why not better, I can already hear you typing? The main reason is that the game on 1.11 is a different beast 1.10 (which itself is quite different from the original 1.0 release).
Namely, the supply simulation is now much more deep and complex, and needs to account for all those railways across the world and the fact that now allies can supply each other.
The game will still die after 1940s, but at least it won't die in the late 1930s because of the new features.

Am I an asshole for absolutely not being impressed?
CKII devs: The game is faster each patch, and our last one was the fastest yet.
EU devs: Ok the game is faster now.
Stellaris devs: Should be 10% faster now.
CKIII devs: Already goes ZOOOM.
HOI Devs: Its not slower, I guess?

Yey.
 
  • 2
  • 1Like
Reactions:
@MatRopert , the U.S. flag in your office is being displayed incorrectly! The stars should be on the left, not the right. Here's what I mean:
If you would kindly, sir, please flip your U.S. flag on the horizontal, this American will pretend this never happened. In the meantime,
I recommend reading this:
You'll be pleased to hear we moved floors this week and the flag gallery has been rearranged with extra care for orientation!
IMG_20211112_183042.jpg
 
  • 5Like
  • 5Haha
  • 2Love
Reactions:
  • 1Like
Reactions:
@MatRopert im curious do u have specific techers for each game or do u tinker on a single overall engine for all the games. how much raw performance is it possible to squeeze out but its ruined by art department or old os support.
 
Ins't NoSQL faster? Might be better for the modders, too - just have to update JSON files.
There are a lot of a few KB files. Reading every single files from the hard drive needs OS involved.
Getting data from one SQL DB needs loading only the DB, not thousands of txt files.

And the modders...
I will discus this example for EU4, because I better know modding it, but in case of HoI4 it's quite similar.
it won't be a problem if Paradox adds modding tools. Now you are writing everything directly in those txt files.
If the game used SQL DB there would be tables. And for example you want to add country.
Now you need to add:
- country file in common/countries
- country history in history/countries
- country TAG to special txt file storing country tags
- country color to special file storing country colors
- country flag file to gfx/flags
- localisation

The DB should have table named countries with:
ID (3 char) - Unique - Country tag
Name - string - country name
Adjective - string - country adjective
- Color1, Color2, Color3 - RGB colors #rrggbb (coded in hex)
- culture - string culture of the country
- religion - string religion of the country
and so on.
You can also split this table to create smaller tables, e.g.:
country_colors:
- ID - country tag
- Color1
- Color2
- Color3

And the modder in contemporary game should have no need to edit bare txt/sql files. They should hav tools doing it. For example tools adding country, where you can add all of this attributes. Some fields like country name and adjective will be typed directly, some of them will be choosed from list (for example religions, and cultures), you should have ability to pick colors form RGB palette by clicking on the color you would like to add, not by using external tools and copy-paste the RGB code to game files, you should have ability to pick and adjust flag from file or to create flag in editor added to tools (made from predefined templates), not by making the file i external progam like Gimp. Changing map should be also graphical tool. The same with GUI (this is part of EU4 modding I hate myself). You have for example map editing (simple) and GUI editing in newer Paradox games (map in HoI4, GUI in CK3).
Returning to HoI4 its really sad, that we don't have any official tool for making focus trees.