Randomness and Strategic AI

[Hroppa]

Really enjoyed the Game Director chat at the conference. Interesting anecdotes came up about about how efforts to improve the strategic AI can sometimes be worse than having the AI make random choices - a deterministic AI will make the same mediocre choices every time, while an AI making random choices will occasionally make good choices as well as mediocre and bad ones, by chance. A little randomness is the 'magic fairy dust' of AI programming.

The devs almost sounded a bit embarrassed about this. I just wanted to say (though I'm sure there are plenty of people at Paradox who know this already) that random strategies are nothing to be ashamed of. In formal game theory (a branch of mathematics/economics), a randomising strategy is often optimal. To give a basic example: in rock, paper, scissors you should randomise between the options with equal weight - if you're predictable, your opponent can choose the relevant counter. To give another example: footballers taking penalty kicks randomise the area they are aiming at (and this has been shown in studies). I'm sure humans playing multiplayer strategy games also do this (subconsciously).

AI in Paradox games is often most easily exploited when it's at its most predictable. It's easy to imagine that injecting some more randomness could make it 'better' (eg in EU4, especially, its predictability makes it easy to bait the AI into bad battles). The key is in choosing the options AI is randomising between, and the weights it's assigning to each option.

I imagine Paradox's AI programmers are already keenly aware of this, and I suspect the main reason even more randomness isn't involved is deliberately to make the AI more easily manipulated and interpreted by players. If AI is too random, I can imagine it being quite frustrating to play against.

Anyway, just wanted to say I found the discussion interesting!

 

[LetaBot]

It is more the case that if you want to do the right mixed strategy, you would need to determine the right probabilities. In GSG things aren't as clear cut as rock/paper/scissors, so you might want certain strategic options to be less likely than others (due to those strategies having a high risk of failing). Running a full scale Expectiminimax though would take too much CPU.

What you usually see if you look at the game files is that the developers tend to set those weights manually. You then also have mods on the workshop that change those weights to be more of an optimal choice. Calculating better weights is possible, but would take more CPU. Since most of the paradox GSG already suffer from late game lag, they (intentionally) decided to keep things simple.

So the frustration from randomness would be that it might pick bad options way too often. How the Soviet Russia AI managed the frontlines at the release of Man the Guns (HOI4) comes to mind. It was less predictable, but it was worse (and more exploitable).