HoI4 Dev Diary - A Tech Lead's Life

Hello everyone, it's your favourite (and only) tech lead, The French Paradox!

In this diary I'll tell you more about our tech, give some insights into the programmer's job, what we've been up to for the past year (and some change) and offer you a chance to ask any question you may have about tech.

Saying old platforms goodbye

First, with the upcoming 1.11 patch we will be retiring some old Linux and Mac platforms that aren't maintained by their own developers anymore.
This means that starting next update the minimum OS version to run HoI will be Ubuntu 18.04 (on Linux) and macOS 10.11 (El Capitan) on Mac.
There shouldn't be much to worry about though, as the previous versions are older than HoI itself at this point and our telemetry shows almost all of you have upgraded long ago.
For those on Windows who look anxiously at their aging Windows 7, keep calm and carry on, we have no plans to drop support for it (although as a friendly tech person I'd still suggest you consider upgrading ;)).

Why do we make those decisions you may wonder? Two reasons: one maintaining old tech comes with a cost that we would rather spend on improving the game for 99% of the users, and two sometimes it blocks the adoption of new tech that would make our programmers job easier and more efficient.
Without going too much into details, the general idea is that we can only use tech that is supported on all platforms, so basically we're only as modern as the oldest thing we support. If you take the case of Linux for example, the previous Ubuntu release was from 2016, meaning HoI could not use anything released since then (and probably a bit before, as Ubuntu LTS releases usually don't ship with the latest shiny goodies). Some time we can work our way around it, but not always.
Speaking of new tech...

Renderers.PNG


DirectX 11

That's right, the next HoI4 release will join Stellaris and the other more recent PDS games by adding DirectX 11 support.
DirectX 9 will still be the default for now, but you will be able to select another renderer in the launcher settings.

There shouldn't be any visual difference between DX9 and DX11 (and OpenGL for that matter), we have been working on making sure the experience will be the same.
The game should load a bit faster however, especially if you use a lot of mods that bring extra textures and models.
Working remotely for more than a year, we also noticed less issues playing with remote desktop and the like, which might not be a big deal for most of you but was quite appreciable for our devs during development.
Finally it did help us tweaking & fixing the new railways & trains graphics (as most graphics debugging tools have dropped support for DX9).


Performance

We know that performance is always at the heart of every discussion about our games and as a Tech Lead I have been keeping a close eye on it.
So far it looks like our release candidate is roughly on par with the current 1.10 patch, performance wise.

Why not better, I can already hear you typing? The main reason is that the game on 1.11 is a different beast 1.10 (which itself is quite different from the original 1.0 release).
Namely, the supply simulation is now much more deep and complex, and needs to account for all those railways across the world and the fact that now allies can supply each other.
While I am writing those words, our compatibility lab is working hard at running the game on various hardware configurations to give us a more complete picture and perhaps a few suggestions for an updated recommended configuration, as the one you can see on Steam right now dates back from the original release.


20211103_120703.jpg

Toying with a 11th generation i9 and a RTX 3090. Will it take off?

On the topic of measuring performance, the Barbarossa update will include an in-game profiler that you can use to measure how your machine performs.
You should be able to spawn it from the console using the magic line imgui show profiler.

For example here is the current release candidate on my home i7-10700 with a RTX 2080 SUPER:

hoi4_1936_RC.PNG

1936 fresh start, speed 5, Direct 11, vsync off

hoi4_1943_RC.PNG

1943 test save, speed 5, Direct 11, vsync off

As you can see this offers a few metrics that can be collected by toggling the Enable/Disable Collection button at the top.
A few things you can get from it:
  • Render time: the average time to render one frame over the last second
  • Render time excluding present: same as previous but excludes the time spent waiting on the GPU to actually present the frame on screen. The difference is usually due to vsync.
  • Frames per second: the FPS count, you know this one
  • Ticks per second: how many in-game hours were simulated over the course of the last second
  • Last tick: the time it took to simulate the last hour or new day/week/month
  • Last 24 ticks average: same as previous but averaged from the last 24 (or less depending on how much time was spent collecting data)
Next you have a nice graph representation of where (in which system) that time was spent. We use it to quickly eyeball a performance report, especially when external programmer tools are not available (for example when it happens on a non-dev machine).

Final tip: if the profiler doesn't show up with the magic console command, try turning it off and on again using imgui off then imgui on.


A personal pet peeve

Programmers at PDS are rarely just that. Most of us get involved in the game development by offering insights on design, balance or content, and I am no exception.
So now is the time to talk about French Communism.

If you remember the French focus tree from La Résistance, if you go communist you get this guy:

Momo.PNG


Maurice Thorez was historically the leader of the Parti Communiste Français (PCF).
If you walk through some cities in France you can even find streets bearing his name still today.
But he had one characteristic trait: he was a die-hard Stalinist who followed the Moscow line until his death in 1964. He had a city and an institute named after him in the USSR.
On the day the Molotov-Ribbentrop Pact was signed, he is in vacation in the Alps and the PCF changes its stance on Germany and Poland overnight following Moscow's instructions without asking him. When he comes back, he just shrugs and goes along with it.

With that in mind, I felt that it was weird you would keep him as a leader if you do not go for the "Loyalty To Moscow" sub-tree.
I did some research in French sources and found a better candidate to replace him. Behold:

1635946239203.png


René Nicod was a member of the PCF in 1939, but when the party decided to realign itself following the Pact, he and a couple other MPs quit the party and founded the Union Populaire Française (UPF) or French Popular Union in English. They denounced the Pact and supported the government's decision to stand with Poland.
In Barbarossa if you own La Résistance and go either Anti-Fascist Coalition or Loyalty to the cause you will get a new party and leader:

Tree.PNG
Tree2.PNG


Each branch will give a different flavour of Nicod inspired by his published opinions pieces during the Great War (he both supported in the French Army and lost his hand in the trenches but also denounced the war on ideological grounds and hoped the French and German workers would unite in a common cause).

Finally, it turned out to be a bit difficult to find a good portrait for our artists. My initial research in the French National Assembly online database only yielded a low-res blurry picture.
But luckily a small French shop decided to sell its stock of old photos on eBay and his mayor portrait was one of the articles.
We could have stopped there and used the ad's preview picture as a source, but of course we had to buy and ship to Sweden.

So here he is, looking over the HOI corner in our Stockholm office!


IMG_20211102_203447.jpg

IMG_20211102_203534.jpg
And with that, I'll happily take your questions :cool:

PS: a few back we talked about Modding Changes. @Aurelien Delay looked at your feedback and has more insights to share that should help you convert your mod to the new character system.
 
  • 129Like
  • 26Love
  • 12
  • 4
  • 3
Reactions:

Catharsis27

Fun Player
22 Badges
May 11, 2013
113
76
  • Darkest Hour
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Semper Fi
  • Arsenal of Democracy
  • Victoria 2: A House Divided
  • Battle for Bosporus
  • Hearts of Iron IV: La Resistance
  • Stellaris: Distant Stars
  • Stellaris: Apocalypse
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Synthetic Dawn
  • Hearts of Iron IV: Death or Dishonor
  • Hearts of Iron IV: Together for Victory
  • Stellaris: Leviathans Story Pack
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV Sign-up
  • Stellaris
  • Victoria 2
  • Stellaris - Path to Destruction bundle
  • Hearts of Iron IV: Expansion Pass
Very much thanks for the info.

I´ve best experience to run Paradox games (Hoi4 / Stellaris) on Linux, but of course not Ubuntu. Iam using Debian based OS and it seems its far faster(the game) than running the game under windows on the same Computer .

With this "in-game-profiler" it will become possible to check out what kind of hardware/software(OS) would be best to Paradox games....at moment it feels (for me) windows isnt that good.

Does anyone else has 2(or more) OS on his Computer and may tell wich OS works better for Paradox games (especially Hoi4 /Stellaris) (Windows, Ubuntu, Debian, Arch, .....)?

I strongly suggest that you may add the in-game-profiler to Stellaris,too (as hotfix) as soon as possible if possible?

May the paradox tech team perform a profi test on a standard Computer with just different OS ? What kind of OS would be best for hoi4 performance ?

Fun: I bet against the tech team if you install (full)Parrot/Kali [not running it as virtual box] it will perform hoi4 far better than any other OS installed on the same Computer, isnt it?
 
Last edited:
  • 1Like
Reactions:

skAzr^^

Recruit
13 Badges
Feb 21, 2020
1
0
  • Europa Universalis IV
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: Colonel
  • Hearts of Iron IV: Together for Victory
  • Surviving Mars
  • Hearts of Iron IV: Death or Dishonor
  • Age of Wonders III
  • Hearts of Iron IV: Expansion Pass
  • Surviving Mars: Digital Deluxe Edition
  • Shadowrun Returns
  • Hearts of Iron IV: Expansion Pass
  • Hearts of Iron IV: La Resistance
  • Battle for Bosporus
you should have added transnistria as a new province in this update beacuse when romania surrendered basarabia and bucovina to the soviet union the soviets had created the moldovian ssr by releasing transnistria, and so they agitated the communists in basarabia into forcing more the romanian goverment to give basarabia to the soviets.
 

RoarkTenjouin

Private
26 Badges
Aug 9, 2021
12
37
  • Hearts of Iron IV: Together for Victory
  • Island Bound
  • Battle for Bosporus
  • Crusader Kings III
  • Hearts of Iron IV: La Resistance
  • Prison Architect: Psych Ward
  • Prison Architect
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Apocalypse
  • Hearts of Iron IV: Expansion Pass
  • Age of Wonders III
  • Hearts of Iron IV: Death or Dishonor
  • Stellaris - Path to Destruction bundle
  • Crusader Kings II
  • Cities: Skylines - Natural Disasters
  • Hearts of Iron IV: Cadet
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Stellaris
  • Cities: Skylines Deluxe Edition
  • Cities: Skylines
  • Victoria 2: Heart of Darkness
  • Victoria 2: A House Divided
  • Victoria 2
  • Teleglitch: Die More Edition
Laval was technically the PM of France on Jan 1st, 1936 (although not for long, as the failure of the Laval-Hoare plan made him unpopular). As a Frenchman I do understand that it feels weird at first to see his face on the starting screen, as today we remember him much for his role in the Vichy government than for his political career in the interwar period.
As for Daladier, to follow history you would need to have France go for the Front Populaire but fail to keep popular support and have Blum ousted.
And then fail to resolve the Winter War in a satisfactory way, leading to Reynaud becoming PM.

There might be a place somewhere for content mechanics that simulate the French Third Republic's infamous instability, but as you can see it would require more thought and a much more substantial change than just changing a leader when doing a focus.
How about at least make it so that if we start from the '36 start date, we can keep Blum/Laval in charge, but choosing the '39 start date will have Daladier in charge? There are other countries where the '39 leader can't take power if you start the game in '36 (EX: I've played as Uruguay from '36 to '48 to earn that achievement, and there was no event where Gabriel Terra got replaced after he left office IRL in '38; yet if I choose Uruguay for the '39 start date, they have someone else in power), so it's not like there isn't a precedent.
 
  • 2Like
Reactions:

Dux Francorum

Recruit
62 Badges
Jan 19, 2014
4
23
  • Stellaris: Digital Anniversary Edition
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Conclave
  • Cities: Skylines - Snowfall
  • Stellaris
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Hearts of Iron IV Sign-up
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: Colonel
  • Hearts of Iron IV: Field Marshal
  • Crusader Kings II: Reapers Due
  • Crusader Kings II: Horse Lords
  • Stellaris: Leviathans Story Pack
  • Crusader Kings II: Monks and Mystics
  • Stellaris - Path to Destruction bundle
  • Steel Division: Normandy 44
  • Hearts of Iron IV: Death or Dishonor
  • Stellaris: Synthetic Dawn
  • Age of Wonders III
  • Steel Division: Normand 44 - Second Wave
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Humanoids Species Pack
  • Stellaris: Apocalypse
  • Stellaris: Nemesis
  • Europa Universalis IV: Call to arms event
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Europa Universalis IV
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • Crusader Kings II
  • Europa Universalis IV: Res Publica
  • Victoria: Revolutions
  • Victoria 2
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Cities: Skylines
  • Cities: Skylines Deluxe Edition
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV: Pre-order
  • Crusader Kings II: Way of Life
Speaking of very (very) minor issues regarding France, it would be nice if two of the Maginot Fortress divisions could be renamed (we can do it manually of course but it's literally a one letter job on PDX's part).

- "Sarreguimines Div. d'Inf. de Forteresse" should be "Sarreguemines Div. d'Inf. de Forteresse" (Sarreguemines with an E not an I
- "Strassbourg Div. d'Inf. de Forteresse" should be "Strasbourg Div. d'Inf. de Forteresse" (Strasbourg with only a single S in french)
 

MatRopert

HoI4 Tech Lead
2 Badges
May 11, 2018
344
2.165
mropert.github.io
  • Battle for Bosporus
I strongly suggest that you may add the in-game-profiler to Stellaris,too (as hotfix) as soon as possible if possible?
Stellaris already has an ingame profiler (I made it back on my time in it) but it's not as complete as this one. You can show it with debug_stats.
May the paradox tech team perform a profi test on a standard Computer with just different OS ? What kind of OS would be best for hoi4 performance ?
I don't think we've done it to a serious extent, but from anecdotal evidence my hunch is that it's faster to load on Linux/Mac (because Windows/NTFS is bad at loading lots of text files) but once inside the game it shouldn't make much of a difference.
 
  • 9
  • 2Like
  • 1Love
Reactions:

alexman

First Lieutenant
On Probation
18 Badges
Jun 2, 2012
286
285
  • Europa Universalis III Complete
  • Divine Wind
  • Heir to the Throne
  • Rome Gold
  • Surviving Mars: Digital Deluxe Edition
  • Age of Wonders III
  • Surviving Mars
  • Hearts of Iron IV: Cadet
  • Europa Universalis III Complete
  • Stellaris
  • Europa Universalis III Complete
  • Rome: Vae Victis
  • Magicka
  • Cities in Motion 2
  • Europa Universalis III: Chronicles
  • Europa Universalis III
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II
i was wondering as a tech lead whats ur opinion on dx12 and vulkan? considering vulkan can run on all OS even win7 and u can emulate dx11 in proton via steam what benefit would u get on vulkan is it worth pursuing? best thing about vulkan is theres no need to wait for a main thread and reduce all rendertime also freeing up 15% cpu.

slightly off topic, whats a artist opinion on new webp or avif format as it has the same quality as a png/jpg but instead of 2mb they are just 48kb with the same image quality. with 500 pictures and portraits u can cut down file size 70% which will also speed up loadingtime and reduce ram
even more so cpus are getting more threads all the time the speed of decompressing 7zips is 500mb/s just dumping all hoi4 stuff into ram from a file would be even more efficient than reading raw from a hdd. as random read is 20ms 4mb/s but sequential read is 200mb/s vs ram thats 100k mb/s and 0.0002ms
 
  • 1Like
Reactions:

MatRopert

HoI4 Tech Lead
2 Badges
May 11, 2018
344
2.165
mropert.github.io
  • Battle for Bosporus
i was wondering as a tech lead whats ur opinion on dx12 and vulkan? considering vulkan can run on all OS even win7 and u can emulate dx11 in proton via steam what benefit would u get on vulkan is it worth pursuing? best thing about vulkan is theres no need to wait for a main thread and reduce all rendertime also freeing up 15% cpu.

slightly off topic, whats a artist opinion on new webp or avif format as it has the same quality as a png/jpg but instead of 2mb they are just 48kb with the same image quality. with 500 pictures and portraits u can cut down file size 70% which will also speed up loadingtime and reduce ram
even more so cpus are getting more threads all the time the speed of decompressing 7zips is 500mb/s just dumping all hoi4 stuff into ram from a file would be even more efficient than reading raw from a hdd. as random read is 20ms 4mb/s but sequential read is 200mb/s vs ram thats 100k mb/s and 0.0002ms
DX12 and Vulkan bring some new interesting tech to the table, but of little benefit for HoI at the moment. Rendering is clearly not a bottleneck for us. The best gains would be achieved simply by having a dedicated render/input thread like CK3 and Imperator do but that can be done on DX11 (even DX9 technically). At this point it's more of an issue of engine architecture.

As for changing texture format it is theoretically feasible but would take up a bunch of our artists time so unless there's huge gains to be made on loading times I think our efforts will be better spent looking at other directions.
 
  • 8
  • 1Like
  • 1Love
Reactions:

Baja385

Private
42 Badges
Dec 28, 2014
15
9
  • Europa Universalis III: Chronicles
  • Hearts of Iron IV: Together for Victory
  • Victoria 3 Sign Up
  • Stellaris - Path to Destruction bundle
  • Cities: Skylines - Mass Transit
  • Europa Universalis IV: Mandate of Heaven
  • Surviving Mars
  • Hearts of Iron IV: Death or Dishonor
  • Europa Universalis IV: Cradle of Civilization
  • Hearts of Iron IV: Expansion Pass
  • Europa Universalis IV: Rule Britannia
  • Europa Universalis IV: Dharma
  • Cities: Skylines Industries
  • Europa Universalis IV: Golden Century
  • Imperator: Rome
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Ancient Relics
  • Hearts of Iron IV: La Resistance
  • Crusader Kings III
  • Battle for Bosporus
  • Europa Universalis 4: Emperor
  • Europa Universalis IV: Rights of Man
  • Europa Universalis IV
  • Europa Universalis IV: Conquest of Paradise
  • Victoria 2
  • Cities: Skylines
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV: Common Sense
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Cossacks
  • Europa Universalis IV: Mare Nostrum
  • Stellaris
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: Colonel
  • Cities: Skylines - Natural Disasters
  • Europa Universalis IV: Third Rome
  • Europa Universalis IV: Res Publica
  • March of the Eagles
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Art of War
  • Crusader Kings II
Hope you will never drop Win7 support. :D
It really depends on how many players still plays on Winodws 7 machines.
Windows 7 is not supported by MS from January 2020 for home users.
Users have possibility to update to Windows 10 for free.
The newest Windows is Windows 11.
More and more people are going to upgrade to Windows 10 or 11 for multiple reasons: security, hardware upgrade (eg. new laptop with preinstalled Win 10/11), convenience (eg. MS Cloud), and so on.
 

MatRopert

HoI4 Tech Lead
2 Badges
May 11, 2018
344
2.165
mropert.github.io
  • Battle for Bosporus
Are you considering switching from a lot of text files to SQL DB?
I believe it could make the game loading faster.
Yes. But it would make modders life much harder. And we like our mods.
How it's it with this multi-core performance right now?
How many cores and threads can HoI use?
HoI has always been able to use all your cores in systems that can scale computations to an arbitrary number of threads.
In practice over 4 you start seeing diminishing returns because for historical reason there's only so many places where the game can split computations.
 
  • 7
  • 1Like
  • 1Love
Reactions: