HoI4 Dev Diary - A Tech Lead's Life

  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
Showing developer posts only. Show all posts in this thread.

MatRopert

HoI4 Tech Lead
Paradox Staff
4 Badges
May 11, 2018
459
3.705
mropert.github.io
  • Hearts of Iron IV: Death or Dishonor
  • Hearts of Iron IV: Expansion Pass
  • Battle for Bosporus
Hello everyone, it's your favourite (and only) tech lead, The French Paradox!

In this diary I'll tell you more about our tech, give some insights into the programmer's job, what we've been up to for the past year (and some change) and offer you a chance to ask any question you may have about tech.

Saying old platforms goodbye

First, with the upcoming 1.11 patch we will be retiring some old Linux and Mac platforms that aren't maintained by their own developers anymore.
This means that starting next update the minimum OS version to run HoI will be Ubuntu 18.04 (on Linux) and macOS 10.11 (El Capitan) on Mac.
There shouldn't be much to worry about though, as the previous versions are older than HoI itself at this point and our telemetry shows almost all of you have upgraded long ago.
For those on Windows who look anxiously at their aging Windows 7, keep calm and carry on, we have no plans to drop support for it (although as a friendly tech person I'd still suggest you consider upgrading ;)).

Why do we make those decisions you may wonder? Two reasons: one maintaining old tech comes with a cost that we would rather spend on improving the game for 99% of the users, and two sometimes it blocks the adoption of new tech that would make our programmers job easier and more efficient.
Without going too much into details, the general idea is that we can only use tech that is supported on all platforms, so basically we're only as modern as the oldest thing we support. If you take the case of Linux for example, the previous Ubuntu release was from 2016, meaning HoI could not use anything released since then (and probably a bit before, as Ubuntu LTS releases usually don't ship with the latest shiny goodies). Some time we can work our way around it, but not always.
Speaking of new tech...

Renderers.PNG


DirectX 11

That's right, the next HoI4 release will join Stellaris and the other more recent PDS games by adding DirectX 11 support.
DirectX 9 will still be the default for now, but you will be able to select another renderer in the launcher settings.

There shouldn't be any visual difference between DX9 and DX11 (and OpenGL for that matter), we have been working on making sure the experience will be the same.
The game should load a bit faster however, especially if you use a lot of mods that bring extra textures and models.
Working remotely for more than a year, we also noticed less issues playing with remote desktop and the like, which might not be a big deal for most of you but was quite appreciable for our devs during development.
Finally it did help us tweaking & fixing the new railways & trains graphics (as most graphics debugging tools have dropped support for DX9).


Performance

We know that performance is always at the heart of every discussion about our games and as a Tech Lead I have been keeping a close eye on it.
So far it looks like our release candidate is roughly on par with the current 1.10 patch, performance wise.

Why not better, I can already hear you typing? The main reason is that the game on 1.11 is a different beast 1.10 (which itself is quite different from the original 1.0 release).
Namely, the supply simulation is now much more deep and complex, and needs to account for all those railways across the world and the fact that now allies can supply each other.
While I am writing those words, our compatibility lab is working hard at running the game on various hardware configurations to give us a more complete picture and perhaps a few suggestions for an updated recommended configuration, as the one you can see on Steam right now dates back from the original release.


20211103_120703.jpg

Toying with a 11th generation i9 and a RTX 3090. Will it take off?

On the topic of measuring performance, the Barbarossa update will include an in-game profiler that you can use to measure how your machine performs.
You should be able to spawn it from the console using the magic line imgui show profiler.

For example here is the current release candidate on my home i7-10700 with a RTX 2080 SUPER:

hoi4_1936_RC.PNG

1936 fresh start, speed 5, Direct 11, vsync off

hoi4_1943_RC.PNG

1943 test save, speed 5, Direct 11, vsync off

As you can see this offers a few metrics that can be collected by toggling the Enable/Disable Collection button at the top.
A few things you can get from it:
  • Render time: the average time to render one frame over the last second
  • Render time excluding present: same as previous but excludes the time spent waiting on the GPU to actually present the frame on screen. The difference is usually due to vsync.
  • Frames per second: the FPS count, you know this one
  • Ticks per second: how many in-game hours were simulated over the course of the last second
  • Last tick: the time it took to simulate the last hour or new day/week/month
  • Last 24 ticks average: same as previous but averaged from the last 24 (or less depending on how much time was spent collecting data)
Next you have a nice graph representation of where (in which system) that time was spent. We use it to quickly eyeball a performance report, especially when external programmer tools are not available (for example when it happens on a non-dev machine).

Final tip: if the profiler doesn't show up with the magic console command, try turning it off and on again using imgui off then imgui on.


A personal pet peeve

Programmers at PDS are rarely just that. Most of us get involved in the game development by offering insights on design, balance or content, and I am no exception.
So now is the time to talk about French Communism.

If you remember the French focus tree from La Résistance, if you go communist you get this guy:

Momo.PNG


Maurice Thorez was historically the leader of the Parti Communiste Français (PCF).
If you walk through some cities in France you can even find streets bearing his name still today.
But he had one characteristic trait: he was a die-hard Stalinist who followed the Moscow line until his death in 1964. He had a city and an institute named after him in the USSR.
On the day the Molotov-Ribbentrop Pact was signed, he is in vacation in the Alps and the PCF changes its stance on Germany and Poland overnight following Moscow's instructions without asking him. When he comes back, he just shrugs and goes along with it.

With that in mind, I felt that it was weird you would keep him as a leader if you do not go for the "Loyalty To Moscow" sub-tree.
I did some research in French sources and found a better candidate to replace him. Behold:

1635946239203.png


René Nicod was a member of the PCF in 1939, but when the party decided to realign itself following the Pact, he and a couple other MPs quit the party and founded the Union Populaire Française (UPF) or French Popular Union in English. They denounced the Pact and supported the government's decision to stand with Poland.
In Barbarossa if you own La Résistance and go either Anti-Fascist Coalition or Loyalty to the cause you will get a new party and leader:

Tree.PNG
Tree2.PNG


Each branch will give a different flavour of Nicod inspired by his published opinions pieces during the Great War (he both supported in the French Army and lost his hand in the trenches but also denounced the war on ideological grounds and hoped the French and German workers would unite in a common cause).

Finally, it turned out to be a bit difficult to find a good portrait for our artists. My initial research in the French National Assembly online database only yielded a low-res blurry picture.
But luckily a small French shop decided to sell its stock of old photos on eBay and his mayor portrait was one of the articles.
We could have stopped there and used the ad's preview picture as a source, but of course we had to buy and ship to Sweden.

So here he is, looking over the HOI corner in our Stockholm office!


IMG_20211102_203447.jpg

IMG_20211102_203534.jpg
And with that, I'll happily take your questions :cool:

PS: a few back we talked about Modding Changes. @Aurelien Delay looked at your feedback and has more insights to share that should help you convert your mod to the new character system.
 
  • 130Like
  • 26Love
  • 12
  • 4
  • 3
Reactions:
Since Imperator Rome implemented some of CK3 tech like tooltips in tooltips after release, is there a chance for HoI4 to do that? Along with a port of the game to the newest Clausewitz Engine?
Very unlikely. At this point HoI's version of Clausewitz is more than half a decade behind the one used by CK3 and Imperator.
Any change would have to be re-developed for HoI.
Idk if this has been asked yet but is there any plans to bring hoi4 to work natively on the apple silicon chips?
Is Paradox considering making a native version of HOI4 for the M1 Macs at some point in the future at this time? I have one as a productivity machine and tried running HoI 4 on it the other day out of curiosity.
Not at the moment. Mac is already a niche for our userbase, Apple Silicon even more so. Making a native build for an entirely new processor is not something we are likely to do unless the market on PC also follows.
How much more performance is it on normal hardware?
Define normal ;)
 
  • 16
  • 4Like
  • 1Love
Reactions:
I've just checked and my DX version is 12... does that mean I'll be able to take advantage of the DX11 support on release, or not? #FormerlyKnowledgeablePerson
Windows includes several version of DirectX. Most machines today should have 9, 11 and 12. Sometimes 10 too. So no worry there.
are you planning on digitally releasing this picture of réné picot for public use or will you keep it on that white board?
We only own this picture, not the rights to it. I'm not entirely sure those are considered public domain. In any case someone did use the same eBay ad picture on Wikipedia if you want to find it...
Is it true, what I heard from several people, that mines cause lagging in MP games? Or is it just a myth? A programmer should know.
There was a really bad performance spikes with mines. The worst has been fixed in Barbarossa.
 
  • 28
  • 5Love
  • 2Like
Reactions:
Since you're supporting Linux, have you considered supporting Vulkan?
Any chance of implementing a Vulkan renderer? I realize that this doesn't automatically mean that the game will run faster but would be nice to have on Linux (and it also works on Windows). Maybe something to keep in mind for the next version of Clausewitz.
Newer games are looking into it I believe. For HoI and other older games it's unlikely.
 
  • 16
  • 1Like
Reactions:
That would be good. I imagine that a lot of countries will now have a general shortage since generals that are assigned as advisors can't be used in the field anymore.
They can do both. They can advise you and still command in the field.
 
  • 24
  • 9Like
  • 2
Reactions:
Very nice to have a performance diary.

One thing im curious about, how will characters differ in terms of performance compared to now?
You're welcome! And as far as we could see the new character system has a very low impact on performance.
 
  • 15
  • 1Like
  • 1Love
  • 1
Reactions:
Give us Edouard Daladier back too, need more French Leaders.

Oh and maybe Paul Reynaud since his portrait is already in the game, just barely ever seen.
Also having Laval at the start feels quite weird.
Laval was technically the PM of France on Jan 1st, 1936 (although not for long, as the failure of the Laval-Hoare plan made him unpopular). As a Frenchman I do understand that it feels weird at first to see his face on the starting screen, as today we remember him much for his role in the Vichy government than for his political career in the interwar period.
As for Daladier, to follow history you would need to have France go for the Front Populaire but fail to keep popular support and have Blum ousted.
And then fail to resolve the Winter War in a satisfactory way, leading to Reynaud becoming PM.

There might be a place somewhere for content mechanics that simulate the French Third Republic's infamous instability, but as you can see it would require more thought and a much more substantial change than just changing a leader when doing a focus.
 
  • 20
  • 4Like
  • 2
Reactions:
Do you take charge of part of script language, like adding new scripts or fixing bugs on it? Does it take a lot of effort? Many times it seems that modable parts only be added when game designers will use these parts, instead of adding in advance. I know little about HOI modding, but I think amount of work is similar between PDS games.
Script support (used by both modders and our own content designer) is specific to the game. Usually each time we add a new mechanic to the game we give script the bindings it needs to interact with it. It's not super costly.
The tricky part comes when content (or mods) want something specific to a country or focus tree. As a general rule, if something will only be used once, it is usually made in script unless it proves hard/impractical to do with the current script system, at which point it is redesigned as a more generic code feature.
Predicting the future being a difficult job, we sometime end up making a script system that gets popular and should have been a code feature instead, and sometimes it's the other way around, there's a generic code system that turns out to be used in only one concrete case.
 
  • 16
  • 1Love
Reactions:
I have noticed that when I open the production menu, my fps often drops. Do you know why this happens?
Can't say I have noticed it. What's your machine spec? Does it happen even when paused?
I think the best way to improve late-game performance, is to tell the AI to stop creating a new unit template every time they want to add a brigade. Right now, every nation has over a dozen different templates in use at a given time, never updating and deleting them.
Templates aren't a big deal. The main issue is the sheer number of units and combats, and the fact that for historical reasons those 2 sub-sytems cannot use multithreading.
... in what order should I upgrade my hardware?

:Newer/faster CPU; More memory; Faster memory; Better graphics; More graphic memory; Newer/faster SSD?
Get a more recent CPU. Anything like i7 6th or 7th gen (or more recent) should give you decent performance.
For RAM, the main concern is how many chrome tabs do you keep open while playing ;) , so 8+ GB is always good.
SSD will help the game start faster, but shouldn't run much faster once started.
GPU is a least concerned, most nVidia/AMD GPUs from the past decade or so should be good enough.
I am by no means a hardware guy, so if this question sounds stupid, please forgive me.

Would the Paradox loader be able to detect hardware (just as an example: NVIDA GTX vs RTX technology) and adjust performance according to present hardware? I am aware that yet oldest to newest technology would have to be included in the files (and probably increase overall folder sizes). Nevertheless, it would make it possible to adapt performance to present hardware. To what extent is this implemented?
We are capable of detecting your hardware specs (that how we keep statistics on what's the average machine for our players), but there isn't much we can tweak to help.
Unless you are running on a integrated GPU, or a very old mobile dedicated GPU, you could toggle all graphics settings to max and not really feel the difference.
HoI is mostly bounded by how fast it can update the world simulation and that's entirely done on your CPU. Running a more "simplistic" simulation on older hardware would also break MP compatibility so that's not really an option for us either.
 
  • 16
  • 2Like
  • 1Love
Reactions:
İs that a Turkish flag back there?
I love paradox more now as a turk
Once upon a time we ordered flags to decorate the HoI corner, with one for each nationality on the team.
The Turkish flag is in the honor of one ex-programmer, the same one who originally made the railways and trains for Barbarossa.
Is "Ticks per second" calculated from the last 24 ticks average? Does is include the longer daily ticks? What about weekly and monthly ticks?

If we have 31 ticks per second like in the image is it correct to assume that 59 days (1.1.1936 12:00 - 1.3.1936 12:00) will take 59*24= 1416 ticks and thus 1416/31 = 45 seconds.

And in your 1943 scenario with 17 ticks per second the 59 days would take 59*24/17 = 83 seconds.

Or is it more complicated?
There is some Paradox math involved ;)
The ticks per second is how many ticks were processed over the last second of real time. This is indeed affected by other stuff such as rendering and daily if there was any.
Also, if your ticks per second go under 24, then the average last 24 ticks will include data outside the last second that were not counted, hence the difference.
Is there any chance to get an engine upgrade in the future, to make better use of multithreads, etc.?
The new engine would not help that much in that regads, this is mostly tied to game code and how the simulation is computed.
The biggest gains would be to rework army/combat updates to be able to use multithreading but that's not an easy task as the combat and movement mechanics you've been experiencing since release have constraints that make that impossible (for ex all movement in processed in sequence: germany moves, then UK, then USSR, etc...).
 
  • 10
  • 1Love
Reactions:
I strongly suggest that you may add the in-game-profiler to Stellaris,too (as hotfix) as soon as possible if possible?
Stellaris already has an ingame profiler (I made it back on my time in it) but it's not as complete as this one. You can show it with debug_stats.
May the paradox tech team perform a profi test on a standard Computer with just different OS ? What kind of OS would be best for hoi4 performance ?
I don't think we've done it to a serious extent, but from anecdotal evidence my hunch is that it's faster to load on Linux/Mac (because Windows/NTFS is bad at loading lots of text files) but once inside the game it shouldn't make much of a difference.
 
  • 9
  • 2Like
  • 1Love
Reactions:
i was wondering as a tech lead whats ur opinion on dx12 and vulkan? considering vulkan can run on all OS even win7 and u can emulate dx11 in proton via steam what benefit would u get on vulkan is it worth pursuing? best thing about vulkan is theres no need to wait for a main thread and reduce all rendertime also freeing up 15% cpu.

slightly off topic, whats a artist opinion on new webp or avif format as it has the same quality as a png/jpg but instead of 2mb they are just 48kb with the same image quality. with 500 pictures and portraits u can cut down file size 70% which will also speed up loadingtime and reduce ram
even more so cpus are getting more threads all the time the speed of decompressing 7zips is 500mb/s just dumping all hoi4 stuff into ram from a file would be even more efficient than reading raw from a hdd. as random read is 20ms 4mb/s but sequential read is 200mb/s vs ram thats 100k mb/s and 0.0002ms
DX12 and Vulkan bring some new interesting tech to the table, but of little benefit for HoI at the moment. Rendering is clearly not a bottleneck for us. The best gains would be achieved simply by having a dedicated render/input thread like CK3 and Imperator do but that can be done on DX11 (even DX9 technically). At this point it's more of an issue of engine architecture.

As for changing texture format it is theoretically feasible but would take up a bunch of our artists time so unless there's huge gains to be made on loading times I think our efforts will be better spent looking at other directions.
 
  • 8
  • 1Like
  • 1Love
Reactions:
Are you considering switching from a lot of text files to SQL DB?
I believe it could make the game loading faster.
Yes. But it would make modders life much harder. And we like our mods.
How it's it with this multi-core performance right now?
How many cores and threads can HoI use?
HoI has always been able to use all your cores in systems that can scale computations to an arbitrary number of threads.
In practice over 4 you start seeing diminishing returns because for historical reason there's only so many places where the game can split computations.
 
  • 7
  • 1Like
  • 1Love
Reactions:
I'm amused that you said that a portrait for René Nicod was difficult to find because the same photo that you used is already in his wikipedia page.
It is now indeed. It wasn't always the case. They used to have the blurry one from the national assembly archives. It took that shop to put a scan on eBay for it to be pulled on Wikipedia.
 
  • 4
  • 4
Reactions:
@MatRopert , the U.S. flag in your office is being displayed incorrectly! The stars should be on the left, not the right. Here's what I mean:
If you would kindly, sir, please flip your U.S. flag on the horizontal, this American will pretend this never happened. In the meantime,
I recommend reading this:
You'll be pleased to hear we moved floors this week and the flag gallery has been rearranged with extra care for orientation!
IMG_20211112_183042.jpg
 
  • 5Like
  • 5Haha
  • 2Love
Reactions: