Could we make T4 dwelling and summon only please ?

[Gloweye]

except it's not really that simple...

Lets suppose you have an input node for every hex on the map. On a small map, thats in the lower 1000's. However, it's not a boolean variable, a node always has a theme, an overlay, may have a treasure site, may have roads, and may have a stack of 1 to 6 units. Each unit may have mystical city upgrades, and may have any of 4 experience ranks, as well as a champion rank which increases it's health. You'll be megabytes in RAM only for the very first layer of your network. Even if at turn 1 most will be unexplored (and thus, empty ? but you can infer information...), late game big maps will require a RAM beyond anything we've ever seen on cosumer hardware. For the first layer of your network. And no, reducing map size is not an option.

That's an amount of information complexity that you're not going to be able to handle with a neural net.

Current AI works by setting a target (like a city the AI wants to capture). Then every induvidual stack will either go towards that target, or take other battles in might want, like clearing treasure sites in an owned city's domain if it thinks it can win that battle. Cities look at the available resources and decide to go build something based on that. If it detects a threat to a city, it'll try to move troops towards it. That's basically all the current AI has to worry about, and it works suprisingly well.

An all-encompassing neural net just isn't going to work. There's to much data.

Also, autonomous vehicle isn't really a comparison. Most of what it does is check for the marks that make up a road, and stay between then, then make a couple distance checks, and worse case do some navigation. It's not really all that complex. AoW is.

 

[Leyrann]

At no point is the network aware of the consequence of its actions.

That's gonna be a problem.

A self-driving car doesn't need to be "aware of the consequences"; all consequences are a direct and obvious result of an action. In Age of Wonders, however, you could do an action that may take 10 or 20 turns, or even longer, to truly take into effect.

Additionally, the AI will undoubtedly come across circumstances it hasn't seen when learning, precisely because of the complexity of the game. There's just too many possible things to happen to all feed to a learning AI and then teach it how to deal with them. There will always be situations unlike anything that has ever happened before.

 

[Nerdfish]

except it's not really that simple...

Lets suppose you have an input node for every hex on the map. On a small map, thats in the lower 1000's. However, it's not a boolean variable, a node always has a theme, an overlay, may have a treasure site, may have roads, and may have a stack of 1 to 6 units. Each unit may have mystical city upgrades, and may have any of 4 experience ranks, as well as a champion rank which increases it's health. You'll be megabytes in RAM only for the very first layer of your network. Even if at turn 1 most will be unexplored (and thus, empty ? but you can infer information...), late game big maps will require a RAM beyond anything we've ever seen on cosumer hardware. For the first layer of your network. And no, reducing map size is not an option.

That's an amount of information complexity that you're not going to be able to handle with a neural net.

Current AI works by setting a target (like a city the AI wants to capture). Then every induvidual stack will either go towards that target, or take other battles in might want, like clearing treasure sites in an owned city's domain if it thinks it can win that battle. Cities look at the available resources and decide to go build something based on that. If it detects a threat to a city, it'll try to move troops towards it. That's basically all the current AI has to worry about, and it works suprisingly well.

An all-encompassing neural net just isn't going to work. There's to much data.

Also, autonomous vehicle isn't really a comparison. Most of what it does is check for the marks that make up a road, and stay between then, then make a couple distance checks, and worse case do some navigation. It's not really all that complex. AoW is.

That's where ConvNet comes in. A situation where a stack is 7 hexes east of a treasure site is essentially the same situation, regardless where the stack and the treasure site is located on the actual map. ConvNet extract this kind of features from the map and quickly reduce the next layer down to several hundred features. This is how image processing network and your eye works. can you imagine each pixel on your megapixel camera acting as an input node ? Now the first layer look for little patches of invariant features. the next layer looks for patches the the space formed by those features, etc. AOW does not contain nearly as much information as the average photo. Yet you can fit even high resolution photos on your graphics ram.

Even the largest AOW maps does not require 1920 X 1200 X 24 bit of color information per channel. The Map state is completely described by the save file, and that file is never more than a couple mags - smaller than most uncompressed bitmaps.

Autonomous vehicles are more complex than any game AI, they drive BETTER THAN HUMAN.

 

[TheDungen]

making t4 summon and dwelling only would make t3 into the staple you mentioned.

 

[Nerdfish]

That's gonna be a problem.

A self-driving car doesn't need to be "aware of the consequences"; all consequences are a direct and obvious result of an action. In Age of Wonders, however, you could do an action that may take 10 or 20 turns, or even longer, to truly take into effect.

Additionally, the AI will undoubtedly come across circumstances it hasn't seen when learning, precisely because of the complexity of the game. There's just too many possible things to happen to all feed to a learning AI and then teach it how to deal with them. There will always be situations unlike anything that has ever happened before.

That's right, it won't play perfectly, but it will adapt to what you do. if you beat it, YOU become the training set and the AI will adjust the network to emulate you.
So there isn't a point where you figure out the AI and it become trivial. Like when you figure out that AI cannot deal with a combination of scorched earth and spell Jammer,
So you can just run around the map erasing their entire empire with AI forever unable to catch your hero stack.

 

[Nerdfish]

making t4 summon and dwelling only would make t3 into the staple you mentioned.

But there are multiple kind of T3 - there are 6 racial T3 in AOW3 and 2 class T3 per class, plus the dwelling ones
A T3 game is an interesting game. A T4 game is an game of mono-stacks.

 

[BloodyBattleBrain]

Hmmm, it seems some are saying it's impossible to create a super duper good AI because it's impossible to calculate all variables.

Well Humans don't calculate all variables, and we're pretty good at this game!

My question to you 2 experts is:

Could an AI be taught to be better than the current AoW3 AI?

I happen to think the current AI is pretty good, but falls apart at higher level play (I.em by the time you are ready for Emperor, the game is actually easier because it's the same routines more or less as Lord, just with more stuff) do could we take the current AI, get players to abuse it a bit, and *update* the AI periodically. Say, every 3 months or so, analyse the data and see that there's an exploit, e.g. I leave a 6 stack 3 hexes from a city but the AI thinks the city is empty and sends a single hero to capture it, even though it actually legitimately sees the 6 stack when approaching the city.

works every time.

Could the AI be taught to analyse the area around the city and avoid such traps?

 

[Leyrann]

Autonomous vehicles are more complex than any game AI

I find that hard to believe.

Input an autonomous vehicle needs:
-Position of lines on the road.
-Local maximum speed.
-Position of nearby objects.
-Speed of nearby objects.
-Recognizing traffic signs or having a database that tells it who gets to go first.
-Recognizing traffic light colours.

Output an autonomous vehicle needs:
-Gas.
-Brake.
-Turn.

Now, I could try to get started on a similar list for AoW, but I'm afraid I wouldn't be done by nightfall.

 

[Bob5]

Hmmm, it seems some are saying it's impossible to create a super duper good AI because it's impossible to calculate all variables.

Well Humans don't calculate all variables, and we're pretty good at this game!

My question to you 2 experts is:

Could an AI be taught to be better than the current AoW3 AI?

I happen to think the current AI is pretty good, but falls apart at higher level play (I.em by the time you are ready for Emperor, the game is actually easier because it's the same routines more or less as Lord, just with more stuff) do could we take the current AI, get players to abuse it a bit, and *update* the AI periodically. Say, every 3 months or so, analyse the data and see that there's an exploit, e.g. I leave a 6 stack 3 hexes from a city but the AI thinks the city is empty and sends a single hero to capture it, even though it actually legitimately sees the 6 stack when approaching the city.

works every time.

Could the AI be taught to analyse the area around the city and avoid such traps?

In theory, it's definitely possible for AIs to get significantly better than humans in Age of Wonders. It's possible to make them unbeatable without any cheating bonuses for the AI. In practice however, it requires a LOT of training for such AIs to actually get to a good level where they're not just fumbling around helplessly. Once self-learning AIs actually 'get it', they can get really good really quickly and become unbeatable, but the time it takes for it to 'get it' depends on how much data it has (how many times it gets to try, how many games it can observe, etc). And that's where the big clutch is, you need a very large amount of games to even really get anywhere, and the question is whether or not that is feasible. To analyse data from players, you'd need to actually acquire that data on your servers, which might fall under privacy laws and such. I've got no idea how that even works legally. Is someone's playing style 'personal data'? I'd imagine it might be.

I'm also not sure if it's easily possible to constrain such AIs to certain difficulty levels without completely crippling them. Self-learning AIs essentially program themselves based on experience, but developers wouldn't really know how such an AI 'thinks' after a while. How do you constrain that properly? And a game like Age of Wonders needs lower difficulty levels too, not everyone is a pro-player, and players need to be able to actually win games against the AI too.


In regards to your question, you're more thinking in line of the traditional AI that follows pre-set routines programmed by developers. Age of Wonders 3 uses this kind of AI, and I'm fairly certain that PlanetFall will do too. Can they be improved on that same path? Probably, there's always a bit room of improvement, however it's difficult to do that without gimping it somewhere else. Ideally you'd want the AI to scout the general area surrounding the city and use that information to make an informed decision on whether or not it should attack. It's not that easy to program, it needs to take into account the movement speeds of your units (how fast can your reinforcements reach the city), their relative strengths compared to yours, etc. It's not unfeasible, but not easy either. Scouting is particularly difficult for AIs I'd imagine. Judging where to scout, which locations to watch, it depends on what you watched in the past couple of turns and where the enemy might have moved in the meantime. It depends on what movement options the enemy has (for instance Sorcerers have a lot of floaters, if it's up against that it might want to watch waters more carefully), it's not all that easy.

 

[BloodyBattleBrain]

Back on topic, I believe that the mod system is designed to make t1 and t2 units more viable into the end game, a little bit like Starcraft I think, where marines are still useful in end game.

I think the core complaint of the OP is having mono stacks, and there are many ways to encourage diverse force composition.

For example, I was thinking about flying units in the new game. We know they have true flying, meaning melee units can't touch them, so there's at least 1 unit (Kir'ko infantry) that's hard countered in a way that doesn't happen in AoW3.

So, how do you balance all flying stacks?Flying by definition means better mobility, and all flying stacks were an issue in AoW3 too (although they tended to be either expensive or fairly weak, like Exalted or Succubi.)

How do you avoid players just making all flying stacks?

I know as Theocrat one of my goals is to get a 6 stack of Exalted and just cause havok .

So, in Planetfall, why not make flying units unable to capture sectors?

It's a bit harsh, a bit arbitrary, but this would make sure players woukd always need marines.

Sinilar thinking coukd be applied to T4 units.

You could:

1. Make them require lots of resources/upkeep
2. Make them require a special resource
3. Have a long tech requirement
4. Give them a distinct weakness and strength (air galley were flying and ranged, but could be brought down by a few archers)
5. Make them mechanically unable to be stacked (I *think* the classic air galley was like this.)
6. Boost their synergy with lower tier units (Shrines power up off devout , but aren't devout themselves, which makes the best Theocrat army actually 5 Exalted and a Shrine)
7. Production/resource overflow to encouragelower tier unit production. In AoW3, no matter how high your production, and I've seen cities with 100 plus hammers, you can produce precisely one unit max per turn. That means my choice in a 100 hammer town comes down to a t4 in 3 turns or a t3 and a half in 3 turns. If it was more like 1 T4 unit OR 6 t1 units, that would be a game changer.

The core issue though is that you only have 6 units in an army, and assuming the basic adjacent hex system hasn't been changed, you'll have 4 armies against 3, or 3 against 4, in any serious battle.

In other words space is at a huge premium.

IF the adjacent hex rule is expanded (as I would like it to be) to include 2 or 3 extra hexes, OR armies exert a zone of control, area determined by army size (larger army = larger zone of control) then you have incentives to have larger armies, which would therefore, with some economic tweaks, mean more lower tier armies.

 

[BloodyBattleBrain]

In theory, it's definitely possible for AIs to get significantly better than humans in Age of Wonders. It's possible to make them unbeatable without any cheating bonuses for the AI. In practice however, it requires a LOT of training for such AIs to actually get to a good level where they're not just fumbling around helplessly. Once self-learning AIs actually 'get it', they can get really good really quickly and become practically unbeatable, but the time it takes for it to 'get it' depends on how much data it has (how many times it gets to try, how many games it can observe, etc). And that's where the big clutch is, you need a very large amount of games to even really get anywhere, and the question is whether or not that is feasible. To analyse data from players, you'd need to actually acquire that data on your servers, which might fall under privacy laws and such. I've got no idea how that even works legally. Is someone's playing style 'personal data'? I'd imagine it might be.

I'm also not sure if it's easily possible to constrain such AIs to certain difficulty levels without completely crippling them. Self-learning AIs essentially program themselves based on experience, but developers wouldn't really know how such an AI 'thinks' after a while. How do you constrain that properly? And a game like Age of Wonders needs lower difficulty levels too, not everyone is a pro-player, and players need to be able to actually win games against the AI too.

Simple!

Opt in telemetry. You're free to use data from how I play to train the AI.

difficulty levels: Have what you have now , PLUS the option to have a trained AI.

And the sample size doesn't, imho, need to be millions of games or whatever.

I think there are enough games now to make a difference.

Remember the goal isn't to make a Deep Blue for AoW3/Planetfall, it's to make an AI that is better than current.

And that evolves/is updated to provide a bit of variety.

Hell, you could just observe the 10 best mp players and copy their moves and it's be a start.

 

[Bob5]

I don't think it's as easy as you make it sound. I know the basics but I don't have any actual experience with AI, I don't know how many games you'd need to get an AI that can actually play the game at a semi-competent level. I'm not sure if it'd work to start where it is now, the two systems might be completely incompatible. I'm not sure how possible or feasible it is with systems like that to give them a head-start in their learning curve. If it has to start all the way at the beginning it'd essentially have to learn how not to get crushed by raiders in the earlier turns, how to move its stacks, and all that stuff. And to get it from that stumbling point onwards all the way to playing competently better than what it is now (essentially Knight level) is a long way. But once it gets it it goes so fast to becoming unbeatable, that gap isn't all that wide, that's just learning how to combine more information to improve its estimations. You might already feed it a bit too many games and make it unbeatable, or just not enough games to be self-damaging.

Just copying moves made by players is a start, but the difficulty is getting it to understand context. Why are you scouting there and not elsewhere? Why are you producing that unit and not another one? And then in the next game you build different units. To truly learn that it needs to get the game, it needs to get context and the broader picture, it needs to find patterns, and to find those patterns it needs a lot of games. Maybe not millions, maybe hundreds or maybe thousands, I honestly don't really know how to estimate that, but it needs quite a bit because Age of Wonders is such a complex game.

 

[TheDungen]

It'd take years just to write the program for how the game learns from these matches. This is not as simple as comparing two pictures to each other which is essentially what a home machine learning system is capable of at this point.

And yeah a machine learning system on this level needs millions of games, probably billions of games. A simple compare two pictures learning require thousands if not tens of thousands.

 

[Gloweye]

It'd take years just to write the program for how the game learns from these matches. This is not as simple as comparing two pictures to each other which is essentially what a home machine learning system is capable of at this point.

And yeah a machine learning system on this level needs millions of games, probably billions of games. A simple compare two pictures learning require thousands if not tens of thousands.

As I said before, you'd need more games than have been played of AoW3. Training set required is to large, and the amount of input data is to many.

 

[Nerdfish]

It'd take years just to write the program for how the game learns from these matches. This is not as simple as comparing two pictures to each other which is essentially what a home machine learning system is capable of at this point.

And yeah a machine learning system on this level needs millions of games, probably billions of games. A simple compare two pictures learning require thousands if not tens of thousands.

Image classifiers are trained using standardized databases. The same approaches can work here having the AI trained on replay databases. It does not require millions of matches, because it can treat every turn as an independent board state. If a single game last 50 turns then it takes 20,000 games to reach a million data points. it should be a simple matter to wait for that many replays to be uploaded to the database. "Opt in telemetry" as Brain pointed out, could provide that many games possibly in one month. (100 games from 200 players each).

As for actual coding, I have being writing AI in tensorflow for the last year. The lion's share of work is designing a representation for the game world (input layers) and make sure gradient flow from output to input. It'd take years for one person, or a couple years for 2 person, or one year for a team.

There is nothing fundamentally insurmountable. The real issue is that it's expensive and people are lazy.

 

[Leyrann]

If a single game last 50 turns then it takes 20,000 games to reach a million data points. it should be a simple matter to wait for that many replays to be uploaded to the database.

But in turn 1 you'll have to take very different actions from turn 50. Like, incomparably different.

 

[Nerdfish]

Just copying moves made by players is a start, but the difficulty is getting it to understand context. Why are you scouting there and not elsewhere? Why are you producing that unit and not another one? And then in the next game you build different units. To truly learn that it needs to get the game, it needs to get context and the broader picture, it needs to find patterns, and to find those patterns it needs a lot of games. Maybe not millions, maybe hundreds or maybe thousands, I honestly don't really know how to estimate that, but it needs quite a bit because Age of Wonders is such a complex game.

It's arguable that AI understand anything. Getting AI to explain why they do things is a nontrivial problem. Maybe they are producing this unit and not another one because a player in the training set produced that unit in a similar situation and not that other one.

 

[Nerdfish]

But in turn 1 you'll have to take very different actions from turn 50. Like, incomparably different.

But the turn 1 board states will also be very different from turn 50 boards. you and the AI can make decision according only to the current board state without caring about which turn it is.

 

[TheDungen]

Image classifiers are trained using standardized databases. The same approaches can work here having the AI trained on replay databases. It does not require millions of matches, because it can treat every turn as an independent board state. If a single game last 50 turns then it takes 20,000 games to reach a million data points. it should be a simple matter to wait for that many replays to be uploaded to the database. "Opt in telemetry" as Brain pointed out, could provide that many games possibly in one month. (100 games from 200 players each).

As for actual coding, I have being writing AI in tensorflow for the last year. The lion's share of work is designing a representation for the game world (input layers) and make sure gradient flow from output to input. It'd take years for one person, or a couple years for 2 person, or one year for a team.

There is nothing fundamentally insurmountable. The real issue is that it's expensive and people are lazy.

Except the parameters change widely so no you would have to have a mlillion cases of every turn, because there are so many diffrent parameters at any point it the game that it needs to learn

But the turn 1 board states will also be very different from turn 50 boards. you and the AI can make decision according only to the current board state without caring about which turn it is.

Yeah but the computer only recognizes this as more things to learn which exponentially increases the number of data points you will need.

 

[Leyrann]

But the turn 1 board states will also be very different from turn 50 boards. you and the AI can make decision according only to the current board state without caring about which turn it is.

I can look at the turn time and take the conclusion that taking an action may or may not be a good idea because of that. If you want the AI to be able to do the same it needs to be aware of that, meaning that "turn" is a data point, meaning you need 50 times more data sets to describe a 50 turn game or, in other words, can't divide games needed by turns per game.