More philosophy
Stoner said:
So, if I understand you correctly, you argue that as long as Hum vs AI and AI vs AI results both yield a logarithmic curve, there's actually little fundamental difference in how the AI plays and how a human plays. I'm with you 100% so far, and to take it one step further this implies that as long as AI and Hum curves are comparable in shape, the AI is WAD, and the rest is basically a game difficulty issue: how much better is the Hum than the AI on average.
Yes. I'm a little uneasy in saying there's "little fundamental difference," but if we qualify that by saying "little fundamental difference
in relation to that particular measurement," it's quite true.
Stoner said:
Further you state that Hum gets better results on average, but medians are comparable. You do not specify, however, why it is you choose to use the median as a measure in this case? Why is it a more pertinent measure than the average? As you say yourself, if you choose to evaluate using the median time, the AI is as good as any human. If you think about it, that statement sort of invalidates the median as a measure of overall AI performance.
Well, let's not over-generalize. I haven't said that medians are ALWAYS best, and so far I have offered just one data point. However, one attractive thing about achieving a stable median value, particularly for measuring time, is that it ensures that
events can work correctly. For example, if I want to start having events representing Soviet partisan activity a few months after the start of Barbarossa, I can (a) have an AI-only event that fires the moment Germany declares war on the USSR, then (b) have a whole pile of partisan events, each with a suitable offset in days, all triggered by the AI-only one having fired. If the median value is right, then half the time the event will fire BEFORE the German AI reaches Leningrad or captures Kiev or whatever, and the other half of the time it will fire AFTER that. Knowing that, I can write a good, relevant event. Or other people can.
A good median value is also helpful in ensuring that the right amount of production occurs -- so for example if Japan's intervention in China drives US consumer goods requirements down, and production up, it is better if Japan takes the proper amount of time to capture half of China, giving the US time to build the carriers and air units and marines needed to go to war in the Pacific. Again, ideally half the time the US build is ahead of schedule, half the time behind.
However, there are some problems with median values, as I will explain . . .
Stoner said:
In your example, the average Hum GER vs AI POL is a tenth of AI Ger vs Hum POL. To me, THAT is the single most important fact, since it clearly implies that the game is way too easy for the average human player. And that is a serious balancing issue in my opinion.
Yes and no. Yes, it was very easy to overrun Poland as Germany in record time in 1.05c. It would be better if the Polish AI could counter the specific "race to Lwow" tactic that most players hit upon. I think the new movement rules in 1.06 will slow the humans down, we'll have to test that.
But we're getting into very tricky territory, because the tasks involved are very short and simple -- take 4-5 provinces as fast as possible. One thing that the AI potentially does better than a human is to work patiently at a long-term, repetitive task like strategic warfare. Doing something in a single, coordinated stroke will always be way easier for a human. I think the fact that the humans couldn't prevent the German AI from completing the conquest of Poland on schedule is more important than the fact that the Polish AI couldn't keep the humans from doing it faster. You try to focus on the things you CAN fix, rather than what maybe bugs you the most.
Stoner said:
What this boils down to, of course, is how should the average HoI player perform against the AI. And that is a matter of taste and difficulty levels. My prayer to you is to not make the mistake of creating a smooth-running mod that is unhistorically easy for the average human player. Looking forward to see your reply on this issue
Hmmm. I think what I'm going to say next answers this, and your concluding questions about objectives in modding.
I don't really see it as a question of letting it be unhistorically easy for the player. I see it more as aiming to make it harder for the player than standard HOI and closer to historical averages in AI-vs-AI. Those two goals are without a doubt achievable, whereas the only way to convince the player he's getting a consistently "hard" game will involve messing up the historicity.
For example, consider the 1939 scenario. In 1939-40, Poland, the Netherlands, Belgium, France, Denmark, and Norway are all doomed if the German player wants to take them. The
time to accomplish these conquests is what we want to look at -- a harder game for the Germans means taking more time. The time is almost certainly going to be distributed logarithmically.
If I use the
average conquest time as my standard, and aim to make the German player's task as hard as it was historically, some weird things happen. First, most games go faster than they did historically. That's just how the curve is shaped. So the experience of the player, who has no idea about the underlying math, is "Hey, this is too easy!" Then occasionally something goes wrong for the Germans and he gets a long period of stalemate against France, or Poland holds the river line -- something. Then the player shrugs and says, "Well, the AI got lucky that time." People don't experience a correct average value as being correct -- they experience it as being mostly too low.
If I used the
median conquest time as my standard, then the player would have a much more convincing sense of the game going as history did, because half the time he would win faster than history, and half the time slower. But what is psychologically right is mathematically wrong, and this is going to have nasty consequences for the AI-vs-AI game,
even if the AI is just as smart as a human!
Why? France fell in six weeks. If six weeks is the
average time for France to fall, then most of the time France will fall faster than that, and a few times it will fall slower, but it will consume about the right amount of effort from the German AI. If six weeks is the
median time, then the
average time will be considerably higher, and that will have an impact on all the other campaigns. What seems just right to the player is wrong in relation to the rest of the game. This is just as true if we're measuring casualties or supply expenditures or air sorties or whatever. And this gap just grows enormously when you have to take into account the weaker performance of the AI.
It's a perverse irony of simulation, one that you won't read about much because very few people have studied it -- it's something I'm planning to include in a book I'm writing. You CAN'T make a simulation that gives a player the "normal" experience of conquering France, because there is no "norm" there to recreate. It looks like there is -- six weeks, 200,000 dead, X number of air sorties, and so on -- but once you start plotting distributions you realize there isn't.
You can give a player the "average" experience, but he will see it as too easy most of the time and then #$%@ hard every so often. That is how wars in general look, when you graph their durations or their casualty totals -- lots of short, small ones, a few long, large ones. Trying to recreate one particular war over and over, we get the same effect.
Another problem which I'm sure you can visualize is that there's a skewing of objectives depending on which side the human plays. If I make the conquest of France take six weeks as the median value, that's "just right" for a German player but it's WAY too easy for the French player. If I make the conquest of France take six weeks as the average value, now I'm right historically, and in my impact on the other parts of the game, but in addition to frustrating the German player (who thinks it's too easy), I'm frustrating the French player (for whom it's usually too hard).
Technically, mathematically, there is no right answer here. If the game results involve logarithmic curves, then you cannot make the game experience "normal" for everyone, or "hard" for everyone, or even consistently "easy" for everyone. Someone has to give up something.
Thus I arrive at the approach I described -- make the AI-versus-AI numbers come out right on average, then look for ways to keep the human performance margin down, in those areas that are critical.
Hope that makes sense. Good discussion, by the way, I'm enjoying this.
