HOI4 Dev Diary - Patch 1.3.3 Update #2

  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

podcat

Game Director <unannounced>
Paradox Staff
12 Badges
Jul 23, 2007
12.811
38.517
  • Europa Universalis IV
  • Hearts of Iron III
  • Semper Fi
  • 500k Club
  • Europa Universalis III: Collection
  • Europa Universalis IV: Pre-order
  • Hearts of Iron II: Beta
  • Europa Universalis: Rome Collectors Edition
  • Mount & Blade: Warband
  • Paradox Order
  • Hearts of Iron IV Sign-up
  • Hearts of Iron IV: Together for Victory
Hi everyone and welcome to the last update on the 1.3.3 patch as we are now getting into final testing of it. This is going to be a big diary with guest writing both from @SteelVolt (AI) and @TomaszKowalczyk (tech lead) as well. Grab a cup of the blackest coffee and enjoy!

Industry - resource lack changes

To continue last weeks theme on industry we have a final change: currently production lines can get a max penalty of -80% due to lack of resources. That effectively mean that a resource lack can be combated by just having more available factories. We felt this was both unrealistic and was making resources less important than we wanted for big nations. Penalties due to lack of resource are now instead calculated by each missing resource over multiple lines and steadily get higher until they can actually get -100% penalty.
resources.jpg



Quality of life!
As normal with patches its not just bug fixes and balance, we've done a bunch of quality of life improvements, what while small do feel big when playing. Lets go:
  • Do you need to start a building construction with top priority? No problem. Now you can CTRL+Click when constructing, so it goes directly to the top of your constructions.
  • Now something that should really be there long ago. When selecting a division, it now scrolls the list of the divisions to show them.
    selelect.jpg
  • Did you have problems to find all your units in a crowdy war? Now there is a button (right bottom side of the screen) with hotkey 'M' that hides ALL counters except your units.
    auto_scroll.jpg
  • Because its sometimes hard to plan ahead over areas where you have little intel we have also added a toggle button to turn off the fog of war effect should you need to to better see terrain etc.
  • We invented a revolutionary new thing which we have decided to call a "loading progress bar"
    load.jpg
  • Do you also have that one friend that is always lagging behind in multiplayer? Got annoyed by the "player lagging behind" message popping up in your face? Now it's an alert. The tooltip explains who is lagging behind and how much. Of course it also automatically disappears when that player catch up.
  • To be consistent with air wings, any unassigned divisions now also get an alert. It's super useful, when for example, a friend is sending you expeditionary forces, and you have no idea where the frell they are. Now just click on the alert to toggle through the unassigned divisions!
    unassigned_divs.png

AI Update!
Several people on the forums have asked for more in-depth details on AI improvements like the ones we did for the Together For Victory release, and you have been heard. Here is an update from @SteelVolt:

Greetings everyone! Taking a short break from otherwise working super hard on improving the Hearts of Iron 4 AI (not counting last week when a flu virus knocked me out), I am once again here to bring you some news about what is going on in my field.

As last time I did these, I thought I would start out with a story from the trenches. This one happened during the current development cycle. After having done some work in several areas of the AI, we noticed in our hands off runs that Germany was fairly consistently defeating England. Getting past the Royal Navy is no small feat, but we initially thought that a Germany that had beat Soviet (which they had started doing at that time) would have industry enough to counter this with airplanes. Turns out that was a bit optimistic. What was really happening was that the naval AI had an issue that made it extremely difficult for it to change regions for assigned fleets, and England had already sent most of their fleet away.

These are the related entries in the changelog:
  • Improved naval AI region evaluation and made it better at reacting more efficiently to changing situations.
  • Naval AI will now attempt to use strike fleets to aid in coastal land combats if they are not on a mission to aid in a naval combat.
The second one being an improvement to how the AI uses its strike fleet (which is the fleets it used not to guard or hunt, but to destroy the enemies main fleet) :)

While the coming patch does not cover all AI improvements in the works, there are several reasons to feel excited about the AI for the next update, and I thought that I would highlight some of the more interesting ones here.

First of all there is the infamous unit spam:
  • Made AI better at calculating supply use when deciding how many divisions it can field.
  • Added equipment_stockpile ai strategy that lets you affect how much stockpile ai feels it needs
  • All nations with many divisions will now attempt to accumulate bigger stockpiles for safety
There were a couple of minor bugs, but also some room for improvements, in the code where the AI estimates how many divisions it thinks it can afford to support. This seems way better now. We also made the AI more careful about its stockpile of equipment as well as making it aim for bigger ones for nations who already had a lot of divisions.
  • Improved deployment AI to manage lines better and improved calculation of desired number of divisions.
Partially connected to the previous point; the AI was way too eager to put a major number of lines working on units but then under certain circumstances just deleting them and starting over. All of this has been given an overhaul, and the AI now acts more moderately both when planning and when reacting to changes.
  • AI will now remove deployment lines for types it has reached its wanted number for.
Really what it says. Before, the AI would keep producing the units it had started working on, which was very inflexible for changing situations.

Moving on, I am surprised this one was not caught earlier:
  • Fixed a serious issue where AI would never counter resistance with area defense orders, completely ignoring conquered territory.
While there had been mentions that the AI could probably do better when it came to countering resistance, it really turned out that it never did this at all. This was due to a code mistake in a single if-statement that made the AI exclusively considered OWNED territory, as opposed to CONTROLLED.

To weather the tightening up and changes we have done to industry has also meant changes for the AI in those areas:
  • Improved logic for when and whom AI should be lend leasing
  • AI is now able to send old equipment as lend lease not just its newest
  • AI can now cancel lend leases it can no longer afford or with nations that no longer make sense
  • AI should no longer be so happy on naval construction over military when it had also alot of civilian industry
The first couple are pretty straight forward: you should be seeing more effective lend leasing now. The last one was an issue affecting many nations (but USA most of all) where having a lot of civilian industry made it think it needed much less military industry in favor of naval construction.

And lastly, the AI is now a lot better at handling safe fronts. It used to sometimes assign significant numbers of units to fronts that posed no threat. Some of these were to countries fighting a common enemy, even when they were winning against said enemy, and others were just...not a threat in any way. Bottom line: more of the AIs units should be making an actual difference now ;)

Lets close with a screenshot that makes me feel pretty happy: Japan not declaring war on the Soviet Union until it has secured the chinese front properly. :)
Untitled.jpg



Performance and Optimizations
This part is for those interested in code details and technical stuff. Just a warning! We figured fans might be interested in more technical details as well as getting some insight in what the job is like for coders. So let us know if this was interesting or made your brains melt! Now I'm gonna hand the word over to our project tech lead @TomaszKowalczyk:

Hi everyone, I'm going to be talking about all the optimization work I did for the patch. We are of course planning to improve it further more in the next patches, however in 1.3.3 the overall speed up is about 14-18%. This may look like not too much, however the main focus was to solve performance "spikes". Those moments when the game literally freezes at a certain moments (usually every 24th hour, when the next day begins). At the very late game date, they were even more annoying than ever. Those I've improved a lot. I was comparing a certain very late save-game, with 1.3.2 and 1.3.3. The first daily tick after hit unpause, in 1.3.2 it took about 4 seconds to advance. In 1.3.3 it was less than ~0.4 sec.

Usually we don't share the nitty-gritty technical details, but this time I'm going to write what and why was slowing down the game:
  • I reworked a bit of the peace conference code that was computing which country can get what. In a gigantic war, where a major country with lots of territory, have lost, pressing SKIP and SKIP and SKIP all over again, was making the game running slower and slower, until almost freeze. This exists in all peace conferences. Just in small wars, it wasn't that badly noticeable. The problem was in a badly nested code loop. Rewriting them worked like a charm.
  • Every time the player click on a navy, and assign the mission in some region(s), the game is doing a "naval path-find" and sends the fleet to go to the necessary area. There was AI logic that wanted to be sure that their fleets were doing the right missions, and it turns out it was spamming "go there", "go there", "go there", while obviously it was enough to tell once. Any kind of path-finding is a pretty complicated and performance heavy thing, but luckily adding a single line of code to stop that solved the problem with the spam!
  • The improvements to the supply system was what had the biggest impact.

    One problem was that plenty of game elements are relying on information about "supplies supported" vs "supplies required". Which is, how many units we can fit in the area before we get the penalty. Every piece of the code that was asking the region "what is the supported and required values?", was recomputing those numbers from scratch. It was asking all units in the area, how high is your supply usage, and how much the region supports (another complicated formula). And it was done hundreds of time by various game elements, in the same computing frame. Instead, I made it so it computes that information only once, and store it in "cached memory". All the game elements now fetch the data directly from cache, which costs almost no performance at all.

    Another problem in the supply systems was the large amount of units. In the very late game, having lots of divisions is a true problem, as it requires lots of computing power. This time it almost killed the supply system. I did plenty of tricks here to improve it. I had to rewrite the way we compute the supplies consumption in each area. Instead of looking at each province and iterating through all the units, asking them "what is your supply consumption value?", I do it from the other side. Every hour, each unit "reports" the supply consumption at it's location, bumping up the overall value in the region. Sounds like same thing, but not really from the technical side. Thanks to that, I could completely remove the super heavy code loop from the supply system. To be honest, that was the nicest optimization ever. How to optimize the function? Remove it :)

    There was so many improvements in the supply system that it doesn't really make sense to list them all, so instead as a summary I will say, that previously the supply system was taking about 65% of the computing time in the daily "spikes". Now it's under 6%. :)
  • Another big optimization to daily "spikes" involved the part of the AI playing around with the division designer to achieve its desired templates. The fix was quite simple here. I've discovered that whenever we try add/remove/replace the division in any slot, the overall template stats are recomputed. That's heavy. Well, maybe not that much heavy, but they became heavy when all countries in the world try almost all possible combinations, in a single frame pass. Turns out, it was enough to refresh the template stats only once after the AI is done filling all the slots.
  • There was also bunch of optimizations in: AI planning naval invasions, AI code responsible for spreading units along the front, the fleets calculating the bonuses from the radars, convoy routes checking which way to go if the certain channels are blocked, and plenty of other hard to explain improvements.
  • There is one last that is my favorite. In our Clausewitz engine, there was an old code loop that nobody ever dared to touch. It was processing all the user interface elements in a "flat manner" instead of the "tree hierarchy". This means, that the more windows and buttons we add to the game, this loop was heavier and heavier. And the windows didn't even had to be shown for it to slow down the game. We always knew about this infamous spot, however reworking it without breaking all the interfaces was nearly impossible. Until now. I found the way! Previously that code loop had ~120 000 passes in each frame, now it's under 700, processing only the necessary interface elements. By that I mean, when you are looking at the technology trees, we are not processing through the hidden production windows and buttons, etc.
As I previously said, we are still working on late-game performance. I've spent only 1,5 week on it so far, and we already get quite much of it so you can expect us to keep doing improvements like this in future patches.

Don't forget to tune in at 16:00CET on www.twitch.tv/paradoxinteractive today where me and Daniel will be continuing our campaign in Kaiserreich for World War Wednesday!
 
Last edited:
To continue last weeks theme on industry we have a final change: currently production lines can get a max penalty of -80% due to lack of resources. That effectively mean that a resource lack can be combated by just having more available factories. We felt this was both unrealistic and was making resources less important than we wanted for big nations. Penalties due to lack of resource are now instead calculated by each missing resource over multiple lines and steadily get higher until they can actually get -100% penalty.
View attachment 241049
Can you elaborate this a bit more?

I understand that the penalty for missing a resource increases based on the amount of how much you are missing overall (not just that production line).
But it leaves many details still open.
E.g.
- Can I still have my high priority lines with full efficiency? (I.e. they get max resources, while other lines are missing)
- If my production line needs e.g. only steel and I have none to give to it what will be the effeciency?
- Is there a difference in the effeciency of one line if there are other lines missing steel tool?
- What if I need 3 resources for my line and have plenty of two but absolutely none of the 3rd in my country? Will I get 100% penalty?
- Etc..


ps. formula would be nice ;)
 
Last edited:
Wow!

From an IT worker's POV, the stuff about both AI and coding fixes was fascinating, so definitely interested to hear more of that in future. :)

Sounds like 1.3.3 should significantly improve the overall experience in Vanilla (particularly at least partially addressing the underlying causes of "division spam").

Three things that it would be really nice to hear about (either in comments or in future diaries):
  • Has the AI been made any better at deciding when it should/should not act on a "casus belli"? (I'm thinking particularly the German, Soviet and UK ones which tend to launch wars the moment they have the opportunity regardless of the risk or opportunity cost)
  • Have the NF trees for the Majors been reworked at all in line with what was done for the Commonwealth ones in TFV, so that there are more logical checks and balances applied in terms of blocking off parts of the tree? (especially the ones that lead to war declarations - Vanilla+ has some very good tweaks to these)
  • Have the issues with divisions getting marooned mid-ocean due to lack of convoys been addressed? (As I understand it this tends to cripple the US war effort)
Great stuff, really looking forward to when this comes out.
 
Fantastic DD. Liking all of it. Thanks for your work guys.

Isn't this resource lack tweak really going to screw over minors and ROTW countries? Or are you also going to tweak worldwide resource distribution a bit?

I took it to mean that the penalty increase the more factories that are on production without needed resources so countries with smaller production runs will be impacted less.
 
I think the resource change is a good, realistic nerf of Germany and the Axis countries in general, at least in a historical WW2 situation where they can't easily trade for resources.
 
Thanks for the diary and your hard work.

I'll echo the above request for me details about how resource lack works.

Also great to hear about optimisation. Will any of the changes you made to supply have behavioural changes that we will notice or is it just about the performance optimisation?

Could you give us a rough idea of how many fewer divisions, say Germany is producing in 1.3.3 compared to 1.3.2?
 
I took it to mean that the penalty increase the more factories that are on production without needed resources so countries with smaller production runs will be impacted less.
If I understand it you get a penalty to all lines that lack the particular resource no matter their priority and that penalty can completely stop your production if you go to negative.
 
@TomaszKowalczyk please tell me that fix to the user interface processing can also be rolled out into the other Clausewitz engine games? Or was it unique to the iteration of Clausewitz that is used in HOI IV ?
 
  • Have the NF trees for the Majors been reworked at all in line with what was done for the Commonwealth ones in TFV, so that there are more logical checks and balances applied in terms of blocking off parts of the tree? (especially the ones that lead to war declarations - Vanilla+ has some very good tweaks to these)
We have done some minor work on a few focuses, but nothing major.
 
If I understand it you get a penalty to all lines that lack the particular resource no matter their priority and that penalty can completely stop your production if you go to negative.

I read it slightly differently, that it increases per factory using the "missing" resource, in priority order - so the first items it gets to on the production screen would get the starting level, and then it would ramp up the more factories were trying to use the "missing" resource until it hit 100%, thus effectively killing production of anything else relying on that resource.

If this is right, it would nicely allow smaller countries to continue to build (at a penalty) whilst stopping industrial behemoths from avoiding resource needs completely via spamming it with more active factories.
 
nice improvements, hoping to get this patch soon ^^.

just don´t quite got the changes in lack of resources penalty.

you mean that each line without Oil will kind of add up until the penalty is too big?
 
Another question that the army list qol change made me think of. Will you implement ways to sort the divisions in an army list the same way we can in the army overview tab?
 
Nice update and many of these fixes will greatly improve the single player experience!

Are there any plans in 1.3.3 to address the convoy allocation issue/bug reported as in several bug reports? Please look at post #530 in this thread:
https://forum.paradoxplaza.com/forum/index.php?threads/the-great-official-ai-thread.988742/page-27

The tool tips in the screen shots seem to show both the importing and exporting countries in a resource transfer assigning convoys to make the transfer. This causes the ai to run out of convoys, prioritize resource and supply convoys over active division overseas transport, and strand divisions at sea. Something is causing ai nations to strand divisions at sea and this might be the root cause?