• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

HoI4 Dev Diary - A Tech Lead's Life

Hello everyone, it's your favourite (and only) tech lead, The French Paradox!

In this diary I'll tell you more about our tech, give some insights into the programmer's job, what we've been up to for the past year (and some change) and offer you a chance to ask any question you may have about tech.

Saying old platforms goodbye

First, with the upcoming 1.11 patch we will be retiring some old Linux and Mac platforms that aren't maintained by their own developers anymore.
This means that starting next update the minimum OS version to run HoI will be Ubuntu 18.04 (on Linux) and macOS 10.11 (El Capitan) on Mac.
There shouldn't be much to worry about though, as the previous versions are older than HoI itself at this point and our telemetry shows almost all of you have upgraded long ago.
For those on Windows who look anxiously at their aging Windows 7, keep calm and carry on, we have no plans to drop support for it (although as a friendly tech person I'd still suggest you consider upgrading ;)).

Why do we make those decisions you may wonder? Two reasons: one maintaining old tech comes with a cost that we would rather spend on improving the game for 99% of the users, and two sometimes it blocks the adoption of new tech that would make our programmers job easier and more efficient.
Without going too much into details, the general idea is that we can only use tech that is supported on all platforms, so basically we're only as modern as the oldest thing we support. If you take the case of Linux for example, the previous Ubuntu release was from 2016, meaning HoI could not use anything released since then (and probably a bit before, as Ubuntu LTS releases usually don't ship with the latest shiny goodies). Some time we can work our way around it, but not always.
Speaking of new tech...

Renderers.PNG


DirectX 11

That's right, the next HoI4 release will join Stellaris and the other more recent PDS games by adding DirectX 11 support.
DirectX 9 will still be the default for now, but you will be able to select another renderer in the launcher settings.

There shouldn't be any visual difference between DX9 and DX11 (and OpenGL for that matter), we have been working on making sure the experience will be the same.
The game should load a bit faster however, especially if you use a lot of mods that bring extra textures and models.
Working remotely for more than a year, we also noticed less issues playing with remote desktop and the like, which might not be a big deal for most of you but was quite appreciable for our devs during development.
Finally it did help us tweaking & fixing the new railways & trains graphics (as most graphics debugging tools have dropped support for DX9).


Performance

We know that performance is always at the heart of every discussion about our games and as a Tech Lead I have been keeping a close eye on it.
So far it looks like our release candidate is roughly on par with the current 1.10 patch, performance wise.

Why not better, I can already hear you typing? The main reason is that the game on 1.11 is a different beast 1.10 (which itself is quite different from the original 1.0 release).
Namely, the supply simulation is now much more deep and complex, and needs to account for all those railways across the world and the fact that now allies can supply each other.
While I am writing those words, our compatibility lab is working hard at running the game on various hardware configurations to give us a more complete picture and perhaps a few suggestions for an updated recommended configuration, as the one you can see on Steam right now dates back from the original release.


20211103_120703.jpg

Toying with a 11th generation i9 and a RTX 3090. Will it take off?

On the topic of measuring performance, the Barbarossa update will include an in-game profiler that you can use to measure how your machine performs.
You should be able to spawn it from the console using the magic line imgui show profiler.

For example here is the current release candidate on my home i7-10700 with a RTX 2080 SUPER:

hoi4_1936_RC.PNG

1936 fresh start, speed 5, Direct 11, vsync off

hoi4_1943_RC.PNG

1943 test save, speed 5, Direct 11, vsync off

As you can see this offers a few metrics that can be collected by toggling the Enable/Disable Collection button at the top.
A few things you can get from it:
  • Render time: the average time to render one frame over the last second
  • Render time excluding present: same as previous but excludes the time spent waiting on the GPU to actually present the frame on screen. The difference is usually due to vsync.
  • Frames per second: the FPS count, you know this one
  • Ticks per second: how many in-game hours were simulated over the course of the last second
  • Last tick: the time it took to simulate the last hour or new day/week/month
  • Last 24 ticks average: same as previous but averaged from the last 24 (or less depending on how much time was spent collecting data)
Next you have a nice graph representation of where (in which system) that time was spent. We use it to quickly eyeball a performance report, especially when external programmer tools are not available (for example when it happens on a non-dev machine).

Final tip: if the profiler doesn't show up with the magic console command, try turning it off and on again using imgui off then imgui on.


A personal pet peeve

Programmers at PDS are rarely just that. Most of us get involved in the game development by offering insights on design, balance or content, and I am no exception.
So now is the time to talk about French Communism.

If you remember the French focus tree from La Résistance, if you go communist you get this guy:

Momo.PNG


Maurice Thorez was historically the leader of the Parti Communiste Français (PCF).
If you walk through some cities in France you can even find streets bearing his name still today.
But he had one characteristic trait: he was a die-hard Stalinist who followed the Moscow line until his death in 1964. He had a city and an institute named after him in the USSR.
On the day the Molotov-Ribbentrop Pact was signed, he is in vacation in the Alps and the PCF changes its stance on Germany and Poland overnight following Moscow's instructions without asking him. When he comes back, he just shrugs and goes along with it.

With that in mind, I felt that it was weird you would keep him as a leader if you do not go for the "Loyalty To Moscow" sub-tree.
I did some research in French sources and found a better candidate to replace him. Behold:

1635946239203.png


René Nicod was a member of the PCF in 1939, but when the party decided to realign itself following the Pact, he and a couple other MPs quit the party and founded the Union Populaire Française (UPF) or French Popular Union in English. They denounced the Pact and supported the government's decision to stand with Poland.
In Barbarossa if you own La Résistance and go either Anti-Fascist Coalition or Loyalty to the cause you will get a new party and leader:

Tree.PNG
Tree2.PNG


Each branch will give a different flavour of Nicod inspired by his published opinions pieces during the Great War (he both supported in the French Army and lost his hand in the trenches but also denounced the war on ideological grounds and hoped the French and German workers would unite in a common cause).

Finally, it turned out to be a bit difficult to find a good portrait for our artists. My initial research in the French National Assembly online database only yielded a low-res blurry picture.
But luckily a small French shop decided to sell its stock of old photos on eBay and his mayor portrait was one of the articles.
We could have stopped there and used the ad's preview picture as a source, but of course we had to buy and ship to Sweden.

So here he is, looking over the HOI corner in our Stockholm office!


IMG_20211102_203447.jpg

IMG_20211102_203534.jpg
And with that, I'll happily take your questions :cool:

PS: a few back we talked about Modding Changes. @Aurelien Delay looked at your feedback and has more insights to share that should help you convert your mod to the new character system.
 
  • 130Like
  • 26Love
  • 12
  • 4
  • 3
Reactions:
Very much thanks for the info.

I´ve best experience to run Paradox games (Hoi4 / Stellaris) on Linux, but of course not Ubuntu. Iam using Debian based OS and it seems its far faster(the game) than running the game under windows on the same Computer .

With this "in-game-profiler" it will become possible to check out what kind of hardware/software(OS) would be best to Paradox games....at moment it feels (for me) windows isnt that good.

Does anyone else has 2(or more) OS on his Computer and may tell wich OS works better for Paradox games (especially Hoi4 /Stellaris) (Windows, Ubuntu, Debian, Arch, .....)?

I strongly suggest that you may add the in-game-profiler to Stellaris,too (as hotfix) as soon as possible if possible?

May the paradox tech team perform a profi test on a standard Computer with just different OS ? What kind of OS would be best for hoi4 performance ?

Fun: I bet against the tech team if you install (full)Parrot/Kali [not running it as virtual box] it will perform hoi4 far better than any other OS installed on the same Computer, isnt it?
 
Last edited:
  • 1Like
Reactions:
you should have added transnistria as a new province in this update beacuse when romania surrendered basarabia and bucovina to the soviet union the soviets had created the moldovian ssr by releasing transnistria, and so they agitated the communists in basarabia into forcing more the romanian goverment to give basarabia to the soviets.
 
Laval was technically the PM of France on Jan 1st, 1936 (although not for long, as the failure of the Laval-Hoare plan made him unpopular). As a Frenchman I do understand that it feels weird at first to see his face on the starting screen, as today we remember him much for his role in the Vichy government than for his political career in the interwar period.
As for Daladier, to follow history you would need to have France go for the Front Populaire but fail to keep popular support and have Blum ousted.
And then fail to resolve the Winter War in a satisfactory way, leading to Reynaud becoming PM.

There might be a place somewhere for content mechanics that simulate the French Third Republic's infamous instability, but as you can see it would require more thought and a much more substantial change than just changing a leader when doing a focus.
How about at least make it so that if we start from the '36 start date, we can keep Blum/Laval in charge, but choosing the '39 start date will have Daladier in charge? There are other countries where the '39 leader can't take power if you start the game in '36 (EX: I've played as Uruguay from '36 to '48 to earn that achievement, and there was no event where Gabriel Terra got replaced after he left office IRL in '38; yet if I choose Uruguay for the '39 start date, they have someone else in power), so it's not like there isn't a precedent.
 
  • 2Like
Reactions:
Speaking of very (very) minor issues regarding France, it would be nice if two of the Maginot Fortress divisions could be renamed (we can do it manually of course but it's literally a one letter job on PDX's part).

- "Sarreguimines Div. d'Inf. de Forteresse" should be "Sarreguemines Div. d'Inf. de Forteresse" (Sarreguemines with an E not an I
- "Strassbourg Div. d'Inf. de Forteresse" should be "Strasbourg Div. d'Inf. de Forteresse" (Strasbourg with only a single S in french)
 
I strongly suggest that you may add the in-game-profiler to Stellaris,too (as hotfix) as soon as possible if possible?
Stellaris already has an ingame profiler (I made it back on my time in it) but it's not as complete as this one. You can show it with debug_stats.
May the paradox tech team perform a profi test on a standard Computer with just different OS ? What kind of OS would be best for hoi4 performance ?
I don't think we've done it to a serious extent, but from anecdotal evidence my hunch is that it's faster to load on Linux/Mac (because Windows/NTFS is bad at loading lots of text files) but once inside the game it shouldn't make much of a difference.
 
  • 9
  • 2Like
  • 1Love
Reactions:
i was wondering as a tech lead whats ur opinion on dx12 and vulkan? considering vulkan can run on all OS even win7 and u can emulate dx11 in proton via steam what benefit would u get on vulkan is it worth pursuing? best thing about vulkan is theres no need to wait for a main thread and reduce all rendertime also freeing up 15% cpu.

slightly off topic, whats a artist opinion on new webp or avif format as it has the same quality as a png/jpg but instead of 2mb they are just 48kb with the same image quality. with 500 pictures and portraits u can cut down file size 70% which will also speed up loadingtime and reduce ram
even more so cpus are getting more threads all the time the speed of decompressing 7zips is 500mb/s just dumping all hoi4 stuff into ram from a file would be even more efficient than reading raw from a hdd. as random read is 20ms 4mb/s but sequential read is 200mb/s vs ram thats 100k mb/s and 0.0002ms
 
  • 1Like
Reactions:
i was wondering as a tech lead whats ur opinion on dx12 and vulkan? considering vulkan can run on all OS even win7 and u can emulate dx11 in proton via steam what benefit would u get on vulkan is it worth pursuing? best thing about vulkan is theres no need to wait for a main thread and reduce all rendertime also freeing up 15% cpu.

slightly off topic, whats a artist opinion on new webp or avif format as it has the same quality as a png/jpg but instead of 2mb they are just 48kb with the same image quality. with 500 pictures and portraits u can cut down file size 70% which will also speed up loadingtime and reduce ram
even more so cpus are getting more threads all the time the speed of decompressing 7zips is 500mb/s just dumping all hoi4 stuff into ram from a file would be even more efficient than reading raw from a hdd. as random read is 20ms 4mb/s but sequential read is 200mb/s vs ram thats 100k mb/s and 0.0002ms
DX12 and Vulkan bring some new interesting tech to the table, but of little benefit for HoI at the moment. Rendering is clearly not a bottleneck for us. The best gains would be achieved simply by having a dedicated render/input thread like CK3 and Imperator do but that can be done on DX11 (even DX9 technically). At this point it's more of an issue of engine architecture.

As for changing texture format it is theoretically feasible but would take up a bunch of our artists time so unless there's huge gains to be made on loading times I think our efforts will be better spent looking at other directions.
 
  • 8
  • 1Like
  • 1Love
Reactions:
Hope you will never drop Win7 support. :D
It really depends on how many players still plays on Winodws 7 machines.
Windows 7 is not supported by MS from January 2020 for home users.
Users have possibility to update to Windows 10 for free.
The newest Windows is Windows 11.
More and more people are going to upgrade to Windows 10 or 11 for multiple reasons: security, hardware upgrade (eg. new laptop with preinstalled Win 10/11), convenience (eg. MS Cloud), and so on.
 
Are you considering switching from a lot of text files to SQL DB?
I believe it could make the game loading faster.
Yes. But it would make modders life much harder. And we like our mods.
How it's it with this multi-core performance right now?
How many cores and threads can HoI use?
HoI has always been able to use all your cores in systems that can scale computations to an arbitrary number of threads.
In practice over 4 you start seeing diminishing returns because for historical reason there's only so many places where the game can split computations.
 
  • 7
  • 1Like
  • 1Love
Reactions: