Hearts of Iron IV - Development Diary 54 - AI

Hearts of Iron IV - Development Diary 54 - AI

  • Pre order now!

    Paradox Development Studio brings you the sequel to one of the most popular strategy games ever made! Crusader Kings III is soon upon us and you can pre order it today!


    Releasing September 1st 2020

SteelVolt

Technomancer
2 Badges
Aug 4, 2014
735
4.199
  • Paradox Order
  • Hearts of Iron IV Sign-up
Greetings every one!

This weeks DD is about something I know many of you have been hoping and waiting for a long time; AI!

"The field of ridiculously high expectations and abysmally low standards." - Wiz

Being the only one crazy enough to want to take on the AI in HoI4 as Wiz went on to other projects, I was thrown in to it over a year ago now, and while it has been the hardest job I have ever had, it has also been the most fun and rewarding one.

So, lets jump in, shall we?

Right off the bat there were a couple of MASSIVE challenges in making the HoI4 AI;
1: Maintaining plausible historical behavior in a relatively dynamic and sandbox oriented game
2: Making the execution of player drawn battle plans solid enough that it would feel beneficial to use it.

Rest assured that significant amounts of sweat, blood and tears have been shed to tackle these issues.

For the first challenge, a couple of things sets the AI of Heart of Iron apart from the other PDS titles. First of all the chronologically compact and recent nature means people have different expectations on the HoI AI to act more or less according to history. While HoI4 is more of a sandbox experience than at the very least its latest predecessor, the set up is still historical and there were often reasons for countries to take certain actions at the times that they did. The challenge here was to have an AI capable of handling new situations dynamically in a way that makes at least some sense, regardless of if they are historical or not.

Our approach to this problem is a dynamic AI as a foundation, with game mechanics pushing countries in certain directions both for players and AI. The primary mechanic for this is, of course, the national foci, which have been covered in previous diaries. These direct the player and AI towards various goals both before and during the great war that eventually happens. But how does the AI pick focuses? This can either be strictly scripted on a per country basis and activated by playing the game in 'historical focus mode'. While it is fully moddable a historical setup for this is included. When that mode is NOT active, the AI picks foci using a combination of scripted weights and dynamic underlying systems that look at the situation of the country.

That dynamic system pushed by scripted weights is something that runs throughout all of the AI. Aside from well over 200 tweakable defines which tweak various aspects of the internal dynamic AI behavior, the weights can pretty much always be tweaked using the trigger system discussed in the modding DD. All in all there are practically no aspect of the AI which cannot be touched through defines, weight tweaks or both.

For example, the desire for Austria to accept the Anschluss event could look like this:
Code:
ai_chance = {
    base = 30
    modifier = {
        add = -25
        GER = { has_army_size = { size < 20 } }
    }
    modifier = {
        factor = 10
        GER = { has_army_size = { size > 19 } }
    }
}
And if they refuse, Germanys attitude towards them can be altered like this:
Code:
anschluss_rejected = {

    # Conditions
    enable = {
        tag = GER
        has_completed_focus = GER_anschluss
        country_exists = AUS
    }
    abort = {
        NOT = {
            country_exists = AUS
        }
    }
   
    # Modification if strategy is enabled
    ai_strategy = {
        type = antagonize
        id = "AUS"           
        value = 300
    }       
}
To give even more flavor to the behavior of countries, there are ways to modify things like the attitude towards other countries with...well, modifiers. These modifiers can be attached to to things like ministers, laws or even leader traits. This means that depending on who is in charge of a country, their interactions with other countries can vary, not just because of their ideology. This is something I want to look at expanding on even more in the future, but the modding possibilities are already notable.

Such a trait may look like this:
Code:
warrior_code = {
    random = no
    ai_call_ally_desire_factor = -40
}
Or this:
Code:
warmonger = {
    random = no
    ai_focus_aggressive_factor = 0.5
}
A couple of areas that gives the modder extra freedom is division template design, where you can tweak target templates, weight which stats the dynamic system should prioritize for specific division types, per country. The dynamic system will even pick up on if they have an unused equipment type and make a template design to accommodate it.
The other noteworthy system is how the AI scores its options in peace deals, which is fully scriptable, both per ideology as well as per country, if you so wish. Big thanks to Groogy for spending the time required on that!

The second big challenge; making the execution of player drawn battle plans good enough to feel useful. Whenever you give over control of a system to the computer, you expose all the weaknesses of the system for player scrutiny. An AI country that makes mistakes with its own troops can often get away with it, but messing up with the players units will lead to frustration.
The solution to this is mainly design; the system is not intended or built to be a magical "win war" button, but rather a tool to organize large numbers of units when every single detail is not critical. Both the AI and the player is expected to oversee critical operations, but they will not have to think about no-brainer situations, and is thus free to handle much much bigger armies and operations than if they had to micromanage them. This means that the internal workings of how the AI handles the units that are part of plans needs fewer bells and whistles, and by extension has fewer things that can go bonkers.
Even so, making it has been as hard as it has been rewarding.

So lets have a peek behind the curtains, into the mind of the AI.
In this situation, Germany has just gone to war against Poland. This tooltip tells us about the general, long term attitude of a country towards other countries. Germany, as we can see, is looking with unkind eyes towards its closest democracies while being supportive towards its fascist faction members.

DD1.jpg

Netherlands, meanwhile, is feeling the heat. Being threatened by Germany, they either want to see it weakened or to be their friend, but do not want to openly antagonize them at the clear risk of being crushed:

DD2.jpg

A more detailed look at Englands attitude towards Netherlands at this time shows us they want to protect the democratic minor of western Europe:

DD3.jpg

But as England has entered the war against Germany moments later, Netherlands is very reluctant to join the Allies, being at clear risk of ending up under the heel of Germany at this time:

DD4.jpg

And lastly, I want to give you a peek at how the AI determines troop levels for orders. Keep in mind that many groups need to share troops, and they may end up with less than they want or need if another group has a much higher priority:

DD5.jpg

Exactly which types of troops end up in which order is also done dynamically, but I have no cool tooltip screenshot of that.

I would say that the proudest moment for any AI developer is when their own creation defeats them in an unexpected manner. For me, such a moment came about two weeks ago when I was playing singleplayer as Germany. I had taken over western Europe easily enough through careful planning and superior weaponry. Wanting to take out England before looking east I was covering guarding my coast in case the Allies got and D-Day ambitions, as well as making sure my border with Soviet was covered, while putting as much resources as I could towards building up my forces for a naval invasion of Britain. It was then that England and USA managed to overwhelm BOTH the Italian and Spanish coastal defenses and gain a solid foot hold. Sending troops to back my friends up, once they arrived both England and USA had landed A LOT more troops, overwhelming both the Italian/Spanish defenses as well as my reinforcements. I could probably have taken on one of them, but both together with such a solid hold in southern Europe was more than I could manage.
Save for that, my best days are when Da9l comes and tells me the AI kicked his ass in one way or another, or just forcing him to change his plans.

A few bullet points that should answer the most burning questions:
* There is no 'hard coding' outside of the scripts. No where in the code does it say "if country is X, then do Y" or similar.
* The dynamic AI systems does not cheat. They use the same information as is available to the player, including estimating enemy forces based on intel numbers.
* Save for boosting party popularity, there is no feature in the game that the AI is completely barred from using. It being intended more as a sandbox feature for players. We have also had to limit it a lot when it comes to staging coups.

Extra kudos goes out to Wiz as head AI honcho, who makes sure I don't fuck everything up, and Groogy, who has gotten to step in and help out when the work load has become too much for one person. I would also like to extend gratitude towards all the amazing beta testers, who have provided tons of valuable feedback and support!

As a closing comment I just want to say that, while no one can hope to meet everyones hopes and expectations of the 'best' possible AI for a game like this, we at Paradox fell it is shaping up really well, and I really hope you all will enjoy playing HoI4 as much, and even more, as I have had making and playing it.
 

Wing_008

Second Lieutenant
13 Badges
Apr 11, 2015
168
211
  • Europa Universalis IV
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Semper Fi
  • Cities: Skylines
  • Europa Universalis IV: Common Sense
  • Hearts of Iron IV Sign-up
  • Stellaris Sign-up
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: Colonel
  • Cities: Skylines - Parklife Pre-Order
  • Cities: Skylines - Parklife
yay a new DD!
 

CharlieFox

Major
63 Badges
Oct 28, 2012
748
833
  • Sword of the Stars
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Heir to the Throne
  • Magicka
  • Europa Universalis IV: Res Publica
  • Victoria: Revolutions
  • Semper Fi
  • Cities in Motion 2
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Crusader Kings II: Monks and Mystics
  • Age of Wonders III
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Cossacks
  • Stellaris - Path to Destruction bundle
  • Crusader Kings II: Reapers Due
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II
  • Cities in Motion
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Europa Universalis IV: Wealth of Nations
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Europa Universalis III: Chronicles
  • Divine Wind
  • Europa Universalis IV
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Hearts of Iron IV Sign-up
  • Cities: Skylines - Natural Disasters
  • Stellaris Sign-up
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: Colonel
  • Europa Universalis IV: Rights of Man
  • Stellaris: Leviathans Story Pack
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Stellaris
  • Crusader Kings II: Horse Lords
  • Pillars of Eternity
  • Crusader Kings II: Way of Life
  • Cities: Skylines Industries
  • 500k Club
Will the AI adapt its production or division templates based on what they know their enemy is building/using?

For instance if as Germany I focus on battleships instead of submarines will UK AI give more weight to producing battleships or carriers?

Likewise, if as Germany I have an infantry only army will Russia AI be more likely to add artillery on their divisions than to add anti tank?
 

skyrimpacman

Sergeant
40 Badges
Jan 5, 2015
61
79
  • Crusader Kings II: Charlemagne
  • Europa Universalis IV: Res Publica
  • Europa Universalis IV: Mare Nostrum
  • Hearts of Iron III
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Art of War
  • Europa Universalis IV
  • Crusader Kings II
  • Crusader Kings II: Sword of Islam
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: The Republic
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Legacy of Rome
  • Prison Architect
  • Crusader Kings II: Conclave
  • Hearts of Iron IV: Cadet
  • Crusader Kings II: Reapers Due
  • Europa Universalis IV: Rights of Man
  • Hearts of Iron IV: Together for Victory
  • Europa Universalis IV: Mandate of Heaven
  • Hearts of Iron IV: Death or Dishonor
  • Age of Wonders III
  • Hearts of Iron IV: Expansion Pass
  • Crusader Kings II: Holy Fury
  • Hearts of Iron IV: Expansion Pass
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: El Dorado
  • Warlock: Master of the Arcane
  • Victoria 2
  • Victoria 2: Heart of Darkness
  • Victoria 2: A House Divided
  • Semper Fi
  • Hearts of Iron III: Their Finest Hour
  • For the Motherland
  • Darkest Hour
Pretty impressive stuff with the AI Paradox. Keep the good work up guys!
 

BoleslavLev

Colonel
47 Badges
Mar 23, 2013
1.087
132
  • Crusader Kings II: Charlemagne
  • Victoria 2: Heart of Darkness
  • Victoria 2: A House Divided
  • Victoria 2
  • Victoria: Revolutions
  • Europa Universalis IV: Res Publica
  • Magicka
  • Leviathan: Warships
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Art of War
  • Europa Universalis III
  • Dungeonland
  • Crusader Kings II: Sword of Islam
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: The Republic
  • Europa Universalis IV
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II
  • Hearts of Iron IV Sign-up
  • Crusader Kings II: Way of Life
  • Hearts of Iron IV: Cadet
  • Crusader Kings II: Reapers Due
  • Europa Universalis IV: Rights of Man
  • Hearts of Iron IV: Together for Victory
  • Europa Universalis IV: Mandate of Heaven
  • Hearts of Iron IV: Death or Dishonor
  • Europa Universalis IV: Cradle of Civilization
  • Hearts of Iron IV: Expansion Pass
  • Europa Universalis IV: Dharma
  • Crusader Kings II: Holy Fury
  • Hearts of Iron IV: Expansion Pass
  • Crusader Kings II: Conclave
  • Europa Universalis IV: Cossacks
  • Europa Universalis IV: Common Sense
  • Mount & Blade: With Fire and Sword
  • Europa Universalis IV: El Dorado
  • War of the Roses
  • Warlock: Master of the Arcane
  • The Showdown Effect
  • Europa Universalis III Complete
  • Europa Universalis III Complete
  • Europa Universalis III Complete
  • Europa Universalis IV: Mare Nostrum
  • Europa Universalis IV: Pre-order
  • Europa Universalis IV: Third Rome
It looks great. I have a question though. In Daniels´ games the AI usually gets pretty gamed. But then he goes to war with all, loses and starts another game. If we´d be able to see the rest of the game and he wouldn´t just keep attacking everybody, would the AI still eventually react to him and maybe beat him?
 
  • 1
Reactions: ObbsiNacho

Marneus Calgar

Corporal
80 Badges
Dec 20, 2015
26
42
  • Crusader Kings II
  • Stellaris - Path to Destruction bundle
  • Crusader Kings II: Monks and Mystics
  • Tyranny: Archon Edition
  • Tyranny: Archon Edition
  • Europa Universalis IV: Rights of Man
  • Stellaris: Galaxy Edition
  • Stellaris: Synthetic Dawn
  • Europa Universalis IV: Mare Nostrum
  • Cities: Skylines - Snowfall
  • Europa Universalis IV: Cossacks
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Res Publica
  • Magicka
  • Europa Universalis IV: Wealth of Nations
  • Ancient Space
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Europa Universalis IV
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Art of War
  • Crusader Kings II: Reapers Due
  • Steel Division: Normand 44 Sign-up
  • Surviving Mars
  • Europa Universalis IV: Mandate of Heaven
  • Cities: Skylines - Mass Transit
  • Cities: Skylines - Natural Disasters
  • Stellaris: Leviathans Story Pack
  • Stellaris: Digital Anniversary Edition
  • Cities: Skylines
  • Europa Universalis IV: El Dorado
  • Magicka: Wizard Wars Founder Wizard
  • Hearts of Iron IV: Field Marshal
  • Hearts of Iron IV: Colonel
  • Hearts of Iron IV: Cadet
  • Stellaris Sign-up
  • Hearts of Iron IV Sign-up
  • Crusader Kings II: Way of Life
  • Age of Wonders: Planetfall - Revelations
  • Stellaris: Galaxy Edition
  • Magicka 2
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Horse Lords
Great!

Indeed!!
 
  • 1
Reactions: Ozamu

SteelVolt

Technomancer
2 Badges
Aug 4, 2014
735
4.199
  • Paradox Order
  • Hearts of Iron IV Sign-up
Will the AI adapt its production or division templates based on what they know their enemy is building/using?
To a degree. As the player, it does not have access to what other countries are building, but to what it encounters in the field.

For instance if as Germany I focus on battleships instead of submarines will UK AI give more weight to producing battleships or carriers?
Given that most naval production is slow, this is determined more through national foci, and the AI the tries to use what it has as well as possible.

Likewise, if as Germany I have an infantry only army will Russia AI be more likely to add artillery on their divisions than to add anti tank?
See question 1 :)
 

HeilLoki

Major
86 Badges
Nov 30, 2010
779
1.389
  • Cities: Skylines Deluxe Edition
  • Hearts of Iron III: Their Finest Hour
  • Impire
  • Europa Universalis IV: Res Publica
  • Semper Fi
  • Cities in Motion
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Crusader Kings II: Reapers Due
  • Europa Universalis IV: Third Rome
  • Heir to the Throne
  • Europa Universalis IV: Pre-order
  • Europa Universalis IV: Cradle of Civilization
  • Stellaris: Galaxy Edition
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Cossacks
  • Cities: Skylines - Snowfall
  • Europa Universalis IV: Mare Nostrum
  • Stellaris
  • For the Motherland
  • Cities in Motion 2
  • Crusader Kings II
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Europa Universalis III: Chronicles
  • Divine Wind
  • Europa Universalis IV
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Call to arms event
  • Hearts of Iron III
  • Hearts of Iron IV: Colonel
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV Sign-up
  • Stellaris: Galaxy Edition
  • Europa Universalis IV: Rights of Man
  • Crusader Kings II: Conclave
  • Crusader Kings II: Horse Lords
  • Pillars of Eternity
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: El Dorado
  • 500k Club
  • Victoria 2
You can micromanage units that are part of an order as much as you want. Planning bonuses are currently not lost as long as the unit remains a part of the order.
Yeah, but after the order is fullfilled, the unit returns to AI control and AI will move it around eventually. Is it possible to tell AI not to move units on its own (aka turning off AI) even if part of a battleplan?
 

D3m0

Captain
75 Badges
Mar 11, 2008
346
592
  • 500k Club
  • Humble Paradox Bundle
  • Victoria 2
  • Europa Universalis III
  • Hearts of Iron III
  • Crusader Kings II
  • Europa Universalis IV: Pre-order
  • Hearts of Iron IV: Colonel
  • Stellaris
  • Darkest Hour
  • March of the Eagles
  • Rise of Prussia
  • Pride of Nations
  • Achtung Panzer
  • BATTLETECH
  • Prison Architect
Will different leaders acting differently be in the game on launch or is this completely something that will only see use in patches or mods post release?
 

SteelVolt

Technomancer
2 Badges
Aug 4, 2014
735
4.199
  • Paradox Order
  • Hearts of Iron IV Sign-up
It looks great. I have a question though. In Daniels´ games the AI usually gets pretty gamed. But then he goes to war with all, loses and starts another game. If we´d be able to see the rest of the game and he wouldn´t just keep attacking everybody, would the AI still eventually react to him and maybe beat him?
In the last WWW it was indeed looking grim for Hungary at the end. They even stated that they were pretty much done for. The AI will pick up on threat generation and typically gang up on common enemies.
 

SteelVolt

Technomancer
2 Badges
Aug 4, 2014
735
4.199
  • Paradox Order
  • Hearts of Iron IV Sign-up
Yeah, but after the order is fullfilled, the unit returns to AI control and AI will move it around eventually. Is it possible to tell AI not to move units on its own (aka turning off AI) even if part of a battleplan?
One of the biggest focuses when designing and working on the "unitcontroller", as we call it internally, is that it should not get in the way of the player, and a player given order must always have priority.

We have been experimenting lately with a system of "stickiness"; if you give a unit a manual order it will stick to that location until front has moved on.

All in all, a combination of using battle plans and giving some detailed orders in critical situations seems to be working very well :)
 

Gort11

Field Marshal
78 Badges
May 22, 2011
4.064
1.686
  • Victoria 2: A House Divided
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Heir to the Throne
  • Leviathan: Warships
  • Magicka
  • Europa Universalis IV: Res Publica
  • Semper Fi
  • Crusader Kings II: Charlemagne
  • Victoria 2: Heart of Darkness
  • Warlock 2: The Exiled
  • Crusader Kings II: Jade Dragon
  • Europa Universalis IV: Third Rome
  • Tyranny - Tales from the Tiers
  • Stellaris: Leviathans Story Pack
  • Stellaris - Path to Destruction bundle
  • Europa Universalis IV: Conquest of Paradise
  • Crusader Kings II
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Divine Wind
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Art of War
  • Europa Universalis IV
  • Europa Universalis III: Chronicles
  • Darkest Hour
  • Europa Universalis III
  • Hearts of Iron IV: Cadet
  • Crusader Kings II: Monks and Mystics
  • Hearts of Iron IV: Together for Victory
  • Warlock: Master of the Arcane
  • Tyranny: Archon Edition
  • Europa Universalis IV: Rights of Man
  • Crusader Kings II: Reapers Due
  • Hearts of Iron IV: Colonel
  • Hearts of Iron IV: La Resistance
  • Hearts of Iron IV Sign-up
  • Stellaris
  • Victoria 2
  • Crusader Kings II: Conclave
  • 500k Club
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Pillars of Eternity
Nice to be able to see what the AI is thinking about in tooltips like that, very cool.