Hearts of Iron IV - Development Diary 54 - AI

[Maykievic]

Well, the Pacific theatre would be a bit dull if the AI couldn't do naval invasions.

I know but coming from hoi 3 when usa and gb never came to continental europe or if it did , it was with just a few units get immediately slaughtered, this is great news

 

[vector1]

All of the defines have descriptions to aid with this.

I appreciate those, but having some elaboration on the logic behind it help us understand the range of values and effect each modifier has. For example, toughness/defensiveness were factored into combat in HOI3, but was not fixed for the longest time since the 0.8 chance to miss should have been 80. Putting it down might help spot errors like that when you say

-- rand(chance to miss/100, toughness) * soft attack = shots hit
or another example
-- AI multiplies WT * factor_1 * factor_2 and if this is less than factor_3 they DOW.

It's much clearer how we should use the modifiers, and what range won't break the code. Any notes will be much appreciated, and thanks for your hard work making HOI4 great!

 

[MonkeyFuzz]

@SteelVolt Can you describe how the AI behavior or abilities may change based on the game's difficulty settings? What might we expect to see on the different levels of difficulty from friendly, neutral, and hostile nations?

Thank you for your DD and your many responses in this thread

 

[agentgb]

will the ai be able to handle paratroopers better then hoi3? in hoi3 the ai never used paratroopers. while the player could, so the ai was left with one less tool in it's toolkit.

 

[SteelVolt]

But what can I script is the question!
What is even an option, aside from NFs and events?
Production "weights"?
Army composition "weights"?
Any option to tell AI where to invest it`s civilian factories? (more military factories/naval yards/civilian industry, ex)
Any options to tell AI what it needs to prioritise during expansion?

A complete list would take too much time and effort right now, and is subject to change (mostly additions), but very few parts are completely walled off. Where values are used to determine things, they are usually available for tweaking.

 

[Invictus5966]

First off let me say that HOI4 fixes everything I didn't like about HOI3. Id play HOI3 over and over and when I got to the invasion of Germany or Russia depending on what nation I played my head exploded...

I am actually working on improving the naval AI right now, but I can't say I have seen that specifically be an issue in HoI4.

Maybe that's partly because in the WWWs naval action has been largely ignored.

Greetings every one!

This weeks DD is about something I know many of you have been hoping and waiting for a long time; AI!

"The field of ridiculously high expectations and abysmally low standards." - Wiz

Being the only one crazy enough to want to take on the AI in HoI4 as Wiz went on to other projects, I was thrown in to it over a year ago now, and while it has been the hardest job I have ever had, it has also been the most fun and rewarding one.

So, lets jump in, shall we?

Right off the bat there were a couple of MASSIVE challenges in making the HoI4 AI;
1: Maintaining plausible historical behavior in a relatively dynamic and sandbox oriented game
2: Making the execution of player drawn battle plans solid enough that it would feel beneficial to use it.

Rest assured that significant amounts of sweat, blood and tears have been shed to tackle these issues.

For the first challenge, a couple of things sets the AI of Heart of Iron apart from the other PDS titles. First of all the chronologically compact and recent nature means people have different expectations on the HoI AI to act more or less according to history. While HoI4 is more of a sandbox experience than at the very least its latest predecessor, the set up is still historical and there were often reasons for countries to take certain actions at the times that they did. The challenge here was to have an AI capable of handling new situations dynamically in a way that makes at least some sense, regardless of if they are historical or not.

Our approach to this problem is a dynamic AI as a foundation, with game mechanics pushing countries in certain directions both for players and AI. The primary mechanic for this is, of course, the national foci, which have been covered in previous diaries. These direct the player and AI towards various goals both before and during the great war that eventually happens. But how does the AI pick focuses? This can either be strictly scripted on a per country basis and activated by playing the game in 'historical focus mode'. While it is fully moddable a historical setup for this is included. When that mode is NOT active, the AI picks foci using a combination of scripted weights and dynamic underlying systems that look at the situation of the country.

That dynamic system pushed by scripted weights is something that runs throughout all of the AI. Aside from well over 200 tweakable defines which tweak various aspects of the internal dynamic AI behavior, the weights can pretty much always be tweaked using the trigger system discussed in the modding DD. All in all there are practically no aspect of the AI which cannot be touched through defines, weight tweaks or both.

For example, the desire for Austria to accept the Anschluss event could look like this:

ai_chance = {
    base = 30
    modifier = {
        add = -25
        GER = { has_army_size = { size < 20 } }
    }
    modifier = {
        factor = 10
        GER = { has_army_size = { size > 19 } }
    }
}

And if they refuse, Germanys attitude towards them can be altered like this:

anschluss_rejected = {

    # Conditions
    enable = {
        tag = GER
        has_completed_focus = GER_anschluss
        country_exists = AUS
    }
    abort = {
        NOT = {
            country_exists = AUS
        }
    }
  
    # Modification if strategy is enabled
    ai_strategy = {
        type = antagonize
        id = "AUS"          
        value = 300
    }      
}

To give even more flavor to the behavior of countries, there are ways to modify things like the attitude towards other countries with...well, modifiers. These modifiers can be attached to to things like ministers, laws or even leader traits. This means that depending on who is in charge of a country, their interactions with other countries can vary, not just because of their ideology. This is something I want to look at expanding on even more in the future, but the modding possibilities are already notable.

Such a trait may look like this:

warrior_code = {
    random = no
    ai_call_ally_desire_factor = -40
}

Or this:

warmonger = {
    random = no
    ai_focus_aggressive_factor = 0.5
}

A couple of areas that gives the modder extra freedom is division template design, where you can tweak target templates, weight which stats the dynamic system should prioritize for specific division types, per country. The dynamic system will even pick up on if they have an unused equipment type and make a template design to accommodate it.
The other noteworthy system is how the AI scores its options in peace deals, which is fully scriptable, both per ideology as well as per country, if you so wish. Big thanks to Groogy for spending the time required on that!

The second big challenge; making the execution of player drawn battle plans good enough to feel useful. Whenever you give over control of a system to the computer, you expose all the weaknesses of the system for player scrutiny. An AI country that makes mistakes with its own troops can often get away with it, but messing up with the players units will lead to frustration.
The solution to this is mainly design; the system is not intended or built to be a magical "win war" button, but rather a tool to organize large numbers of units when every single detail is not critical. Both the AI and the player is expected to oversee critical operations, but they will not have to think about no-brainer situations, and is thus free to handle much much bigger armies and operations than if they had to micromanage them. This means that the internal workings of how the AI handles the units that are part of plans needs fewer bells and whistles, and by extension has fewer things that can go bonkers.
Even so, making it has been as hard as it has been rewarding.

So lets have a peek behind the curtains, into the mind of the AI.
In this situation, Germany has just gone to war against Poland. This tooltip tells us about the general, long term attitude of a country towards other countries. Germany, as we can see, is looking with unkind eyes towards its closest democracies while being supportive towards its fascist faction members.

View attachment 173986

Netherlands, meanwhile, is feeling the heat. Being threatened by Germany, they either want to see it weakened or to be their friend, but do not want to openly antagonize them at the clear risk of being crushed:

View attachment 173987

A more detailed look at Englands attitude towards Netherlands at this time shows us they want to protect the democratic minor of western Europe:

View attachment 173988

But as England has entered the war against Germany moments later, Netherlands is very reluctant to join the Allies, being at clear risk of ending up under the heel of Germany at this time:

View attachment 173989

And lastly, I want to give you a peek at how the AI determines troop levels for orders. Keep in mind that many groups need to share troops, and they may end up with less than they want or need if another group has a much higher priority:

View attachment 173990

Exactly which types of troops end up in which order is also done dynamically, but I have no cool tooltip screenshot of that.

I would say that the proudest moment for any AI developer is when their own creation defeats them in an unexpected manner. For me, such a moment came about two weeks ago when I was playing singleplayer as Germany. I had taken over western Europe easily enough through careful planning and superior weaponry. Wanting to take out England before looking east I was covering guarding my coast in case the Allies got and D-Day ambitions, as well as making sure my border with Soviet was covered, while putting as much resources as I could towards building up my forces for a naval invasion of Britain. It was then that England and USA managed to overwhelm BOTH the Italian and Spanish coastal defenses and gain a solid foot hold. Sending troops to back my friends up, once they arrived both England and USA had landed A LOT more troops, overwhelming both the Italian/Spanish defenses as well as my reinforcements. I could probably have taken on one of them, but both together with such a solid hold in southern Europe was more than I could manage.
Save for that, my best days are when Da9l comes and tells me the AI kicked his ass in one way or another, or just forcing him to change his plans.

A few bullet points that should answer the most burning questions:
* There is no 'hard coding' outside of the scripts. No where in the code does it say "if country is X, then do Y" or similar.
* The dynamic AI systems does not cheat. They use the same information as is available to the player, including estimating enemy forces based on intel numbers.
* Save for boosting party popularity, there is no feature in the game that the AI is completely barred from using. It being intended more as a sandbox feature for players. We have also had to limit it a lot when it comes to staging coups.

Extra kudos goes out to Wiz as head AI honcho, who makes sure I don't fuck everything up, and Groogy, who has gotten to step in and help out when the work load has become too much for one person. I would also like to extend gratitude towards all the amazing beta testers, who have provided tons of valuable feedback and support!

As a closing comment I just want to say that, while no one can hope to meet everyones hopes and expectations of the 'best' possible AI for a game like this, we at Paradox fell it is shaping up really well, and I really hope you all will enjoy playing HoI4 as much, and even more, as I have had making and playing it.

In another thread there is some talks going on about how the AI moves troops into another country and nobody replied to this aspect of it.
The 2 examples that showed up in the last WWW was:

1 - Germany moved dozens of troops into neutral hungry to attack Poland. My question is why and how?
2 - Daniel was able to move troops thru Russian controlled Poland to attack Germany... Same questions.

They weren't allied or at war IMHO Nations shouldn't be able to move into other nations without permission or a DOW


Anyway that's for your time and every ones hard work. Cannot wait til the game comes out.

 

[SteelVolt]

I appreciate those, but having some elaboration on the logic behind it help us understand the range of values and effect each modifier has. For example, toughness/defensiveness were factored into combat in HOI3, but was not fixed for the longest time since the 0.8 chance to miss should have been 80. Putting it down might help spot errors like that when you say

-- rand(chance to miss/100, toughness) * soft attack = shots hit
or another example
-- AI multiplies WT * factor_1 * factor_2 and if this is less than factor_3 they DOW.

It's much clearer how we should use the modifiers, and what range won't break the code. Any notes will be much appreciated, and thanks for your hard work making HOI4 great!

I understand the frustration. Often the thing a factor is multiplied by is not so basic as "WT", but may be something like "WT change over the last month divided by country size if minor and army size if major". I try to be as clear as I can in my descriptions, but we do have staff who actively works with supporting modders, and they should be able to answer questions if something is really unclear when you try to use it.

 

[SteelVolt]

will the ai be able to handle paratroopers better then hoi3? in hoi3 the ai never used paratroopers. while the player could, so the ai was left with one less tool in it's toolkit.

It will at least be able to use them

 

[Invictus5966]

Sorry one more question about the AI.

If I played the USA and I invaded Germany occupied lands would the AI England follow up my invasion with troops and help after they did their plans ect...

 

[Adige]

Last two DD were extraordinary. Also @Guraan and @SteelVolt were super helpfull. Thank you guys.

 

[agentgb]

It will at least be able to use them

excellent!!! thank you very much for the reply!

 

[SteelVolt]

First off let me say that HOI4 fixes everything I didn't like about HOI3. Id play HOI3 over and over and when I got to the invasion of Germany or Russia depending on what nation I played my head exploded...

Maybe that's partly because in the WWWs naval action has been largely ignored.

In another thread there is some talks going on about how the AI moves troops into another country and nobody replied to this aspect of it.
The 2 examples that showed up in the last WWW was:

1 - Germany moved dozens of troops into neutral hungry to attack Poland. My question is why and how?
2 - Daniel was able to move troops thru Russian controlled Poland to attack Germany... Same questions.

They weren't allied or at war IMHO Nations shouldn't be able to move into other nations without permission or a DOW


Anyway that's for your time and every ones hard work. Cannot wait til the game comes out.

The WWW have primarily been about what people have asked to see.

1 - Yeah, there is a known issue with AI drawing front lines along fronts it cannot attack over. I have an issue on it, and it should be a fairly easy fix
2 - If two countries are at war with a common enemy they can move through each others territory. This rule is a design choice completely unrelated to the AI.

 

[tommylotto]

Thanks for taking time to answer our questions SteelVolt. Here is one...

If I were to add a new equipment type (say, heavy artillery) and a new battalion type (say, heavy artillery battalion) and that battalion type, not surprisingly, requires heavy artillery as equipment. Then, if I added a heavy artillery battalion to a nation's template, will the AI automatically see the need for heavy artillery equipment and allocate military factories to satisfy that need or do we need to separately tell the AI what new equipment types to build and in what numbers?

 

[SteelVolt]

Sorry one more question about the AI.

If I played the USA and I invaded Germany occupied lands would the AI England follow up my invasion with troops and help after they did their plans ect...

They _can_, but it is a balancing question if they would or not. Nothing is really stopping them from helping out, though.

 

[SteelVolt]

Thanks for taking time to answer our questions SteelVolt. Here is one...

If I were to add a new equipment type (say, heavy artillery) and a new battalion type (say, heavy artillery battalion) and that battalion type, not surprisingly, requires heavy artillery as equipment. Then, if I added a heavy artillery battalion to a nation's template, will the AI automatically see the need for heavy artillery equipment and allocate military factories to satisfy that need or do we need to separately tell the AI what to new equipment types to build and in what numbers?

Even better: you wouldn't have to add it to a template yourself, the AI would pick up on it automatically and consider it for templates. That would create a need for the equipment, which would go into production.

 

[Gort11]

1 - Germany moved dozens of troops into neutral hungry to attack Poland. My question is why and how?
2 - Daniel was able to move troops thru Russian controlled Poland to attack Germany... Same questions.

I believe the way this works is this:

Normally nobody can move troops into your territory unless they declare war on you.

You are country A, and you declare war on country B. Country C then also declares war on country B. Country C can then move its troops into your territory because you're sharing a war.

 

[Kensai7]

* Save for boosting party popularity, there is no feature in the game that the AI is completely barred from using. It being intended more as a sandbox feature for players. We have also had to limit it a lot when it comes to staging coups.

Well, material for a future DLC. Make the AI intelligent enough to make in rare occurances also political what-if choices.

 

[Invictus5966]

The WWW have primarily been about what people have asked to see.

1 - Yeah, there is a known issue with AI drawing front lines along fronts it cannot attack over. I have an issue on it, and it should be a fairly easy fix
2 - If two countries are at war with a common enemy they can move through each others territory. This rule is a design choice completely unrelated to the AI.

With Number 2 Daniel was completely neutral with Germany (Not in Axis) and not at war with Poland but Germany marched thru Hungry to attack Poland and His troops started to March across Russia to attack Northern Germany when he DOWd Germany. I agree if they are both at war yes but this wasn't the case, was it a bug?

 

[Da9L]

With Number 2 Daniel was completely neutral with Germany (Not in Axis) and not at war with Poland but Germany marched thru Hungry to attack Poland and His troops started to March across Russia to attack Northern Germany when he DOWd Germany. I agree if they are both at war yes but this wasn't the case, was it a bug?

I was at war with the allies and so was Germany aka common enemy and that would grant them the access. I'm also pretty sure Poland was quite quickly invited to the allies and called in against me. Same with Soviet Union, they were at war with the allies and so was I.

 

[Gort11]

It kinda feels like they should still have to ask first