Using OpenAI or similar to potentially improve the AI

[Shinkuro Yukinari]

We've all heard it, we all agree, Paradox is workin on it(we are yet to see their work in practice though)
AI is god-awful.
Has anyone considered letting OpenAI do a couple of games as specific nations, then analyzing the data it compiles for further use?
Give it the present AI files and let it start from there.
Whilst it will take time, it won't be that time-intensive as you just need to let the game run at speed 5 and HoI4 campaigns are much shorter than say EU4. Heck, you don't even need to let it run past like 1944-1945 as that is when the game dulls out and war comes down to who has more fighters so it can nuke the enemy into submission.

 

[Kadanz]

If only it was that easy

 

[jjjdddsss]

I'm not sure how else to put this, but that's not how any of this works. Machine learning doesn't produce anything that would help write algorithmic game AI.

 

[Daddl]

Spoiler

Having five years computer science studies at university, including AI and machine learning courses, I can say that while some overlap between academic AI and game AI exists, the two are very different beasts. The goal of academic AI is (typically) finding automated ways of solving certain problems optimally. The goal of game AI is to entertain. One of the hardest problems for AI is pattern recognition, something humans are very very good at. In a complex strategy game the pattern of a situation includes terrain (rivers, mountains, marshes etc), nuber of units, type of units/equipment, current and potential enemies, current and potential allies, air support, naval support, naval invasions, resource locations etc etc. All this, and more, over a potential area of over 13k provinces plus several off map mechanics.
By comparison, go has 361 positions, with a total of 3 possible states each.

I did eagerly follow the results of AlphaGO, and while I would love to play around with deep learning, beyond board games I do not see it applied in games in the near future. AlphaGO was essentially trained on a supercomputer with a massive set of training data for a turn based games where only one move at a time is possible. HoI 4 being a pausable real time game with hard constraints on time for the AI just completely blows this approach out of the water in terms of complexity. Even if you consider each hourly tick a turn, you can still change a very large number of things within each tick.

So, tl;dr: game AI is usually hand tailored for the game it relates to, as is the case for HoI. There are some generalized mechanics detailed in books, such as how to structure agents and state machines. Deep learning is really cool, and the accomplishments of AlphaGO are amazing, but for computer games in general the current state of it is of very little use.

To get the AI on a level to beat the best humans in HoI like AI can do in Chess, Go or other games you probably need around 20 employees, computing power worth a couple dozen million dollars and 3 years time.

Its not that easy. Be sure someone would do it if it was.

 

[Marcion]

There is already research being done on training AI to play StarCraft and other modern PC games, but it is just research at this point. There's no guarantee that the results with have any practical applications as far as creating algorithms for simulated opponents, and this is bleeding-edge stuff involving top people in the field.

Some day anybody might be able to use an off-the-shelf ML engine to train up a game AI, but that's a ways off.

 

[Magnificent Genius]

Someone made an AI for Dota, that I believe had limited learning capabilities. In a very limited tournament setting, its performance got WORSE as the tourney progressed. Dota is not as complex as HoI. I'm willing to give Paradox a break on the AI.

 

[bwhite9]

Well so far Open AI hasn't been very successful in making AI for games. Last I checked the best it got was best in the world at 1v1. Now I know that might seem impressive but HOI4 is a much more complicated game than Dota and the team working on that AI was probably just as big as the entire HOI4 team. So just running Open AI a few times isn't going to get the team anywhere. Also, the AI that OpenAI would make probably can't be merged with current AI so they would either have to cancel all development of the AI wait till OpenAI is done or have two builds of the AI going at the same time. As well the devs don't really want the whole world to be filled with AI that uses every exploit they do what it to confirm a bit to history. Another issue with using learning AI is that a key step in their development is to test them to see how good they are at whatever it is they are doing. And there really isn't a good test to see how well the AI did.

So it just isn't feasible in any way for it to happen. Maybe for HOI5 when AI programming is much more advanced.

 

[Gyrvendal]

Machine learning has nothing to do with actual intelligence, despite claims to the countrary. It is just an algorithm that optimizes a large number of parameters in order to reproduce/imitate a desired result.

It can automate some tasks that could previously only be performed by a human, but so can a pocket calculator.

It could never work on a game like Hoi.

 

[xsmilingbanditx]

There is this thing about "AI". I do not know the proper translation but there is a scientific field called "game theory". Games like Chess, Go or Tic Tac Toe are "Tree Games". Meaning there ALWAYS exists ONE optimal strategy where one Side will always win and another Strategy where the other side cannot lose. For chess or Go those still have to be found. The complexity is defined by how fast the "decision tree" broadens with each iteration/step/turn.
In these type of games, "AI" will sooner or later beat any human. Period.
HoI or Stellaris are way, way more complex. IF they are in the same category as Chess or Go (which still is open to discussion/research) the complexity would be way higher. Meaning without HUGE computational power the "AI" cannot plan ahead maybe more than one or two "turns" like humans can.
That said, without wanting to sound offensive, I do not understand why everbody complaining about AI here thinks they are more competent than the developers. I had some courses at university about that back in the days and I still would only assume to be able to grasp the sheer complexity of a competitive AI in Grand Strategy Games.
For example, even the "really good " Chess AI would fail hard and would have to relearn EVEYTHING if you would change Chess from 8x8 fields to, say, 9x9 or 7x7. Same goes for the OpenAI if you would change some paramereters of Dota Characters.

About Starcraft: AI is so good there because

a) you so have "build pattern strategies" the first 15 Minutes or so...this is just a numbers game and Computers are made for that
b) AI is great at "micro" and in SC, Battle outcome is pretty deterministic (because damage is)
c) games do not last that long, meaning you do not need to "optimize" for tomorrow, just for the next few seconds and repeat that over and over (which helps with the above)

With all that said, as long as you do not have a Cray SC at home for gaming there will Not be a competitive AI that can plan ahead more than a few "ticks" in complex games.

Well, that is quite a wall if text. I hope I could give you some insights. There is more and I could go into some depths here but typing on my smartphone is a pita

Feel free to comment

 

[Markus Marius]

I think we'll have to wait for Quantum computers ( 5000 qbit or larger ) before we get an AI that can play this type of game well.

 

[xsmilingbanditx]

I think we'll have to wait for Quantum computers ( 5000 qbit or larger ) before we get an AI that can play this type of game well.

For Insane AI, install "Earth 2140" plus the Patch that "unleashes" the AI. It was so good that it wrecked the Devs.
It attacked you and reacted depending on weak spots of your defense and built units to counter. It had incredible "micro" and nee exactly how many units had to shoot one unit to instagib it and so on...