I don't think the solution space or AoW4 battles is larger than starcraft 2, I think it is much smaller. Even if it would be too hard to start reinforcement learning from scratch there are many ways to improve performance and simplify things. Creating simplified scenarios and progressively making them more complex (curriculum learning) is one easy to think of option. If all a unit would have to do is select the "shoot at enemy" action it really won't be hard to learn. Then "move to enemy" and "shoot at enemy". Progressively more abilities, units and map layouts can be added.
Another simplifying factor is that defining the reward function is not very difficult in this problem. There are many smoothed functions to choose from, like counting damage points done and damage taken.
Also you don't have to start over when you release a patch. They don't change the AI every patch now, why would they do it then? Even if you wanted to retrain you just use the existing model as starting point and only some weights have to be adjusted. The same is true when you add an expansion.
I am not saying that from a business perspective it is a good idea to use deep learning to develop the battle AI for AoW4. But you are implying that this is a tremendously challenging problem that just can't be solved. And that is simply not the case.
