Home Trending Latest threads New posts Developer diaries
Jump to latest Follow Reply
  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
Stellaris 51302 Bug Reports 32017 Suggestions 19795 Tech Support 2993 Multiplayer 383 User Mods 4698 Stellaris AAR (After Action Reports) Console edition 1237
Previous dev diary Next dev diary
Show only dev responses

Stellaris Dev Diary #172 - Reworking the AI

Bonjour everyone, it’s the French Paradox speaking! For those who don’t know me, I’ve joined the Stellaris team this December after a year and a half as a programmer on Europa Universalis IV.

Today, we are gonna talk about AI.

pasted image 0.png

A good introduction for those new to the field​

Fifty Shades of AI
There are several AI modules in Stellaris. For historical reasons we call them “ministers” as each one is supposed to handle a specific role in an AI empire.

There are 3 broad kinds:
  • The AI foreign minister handles diplomacy, federations, galactic community, peace deals and the like
  • The AI interior minister is in charge of the economy. He keeps budgets and order constructions, both civil and military.
  • The AI military minister is in command of all troops and military fleets, and also responsible for laying out strategic plans when at war.
For each of those ministries there are different “ministers” there are several options that can be selected for every empire in the game. All of those have generic one which behaves more or less like we’d expect a player to and is used for most AI empires. Then we have a bunch of specialized ones for special tags such as space monsters, fallen empires, crisis, marauders and the like.

As almost everything in our games, AI is configurable in script for our modders, although I’m not exactly sure what would happen if you assigned a space monster military AI to the caravaneers ;)

In guise of a welcoming gift when I joined the team, I was tasked with reworking the military one...

The Military AI
To give you a little bit of background, there were several generations of military AIs in Stellaris. The generic one (used by most “classic” empires) was redone by the great @sidestep last year, while the more specialized ones (crisis, space monsters) have kept close to what they were on release. In the midst of the sad and dark swedish winter, I managed to bring some improvements that I’ll showcase today.

First of all, I worked on visualization to help us debug how the AI “thinks”. Funny thing is, it already made it look “better” to audiences even if it didn’t actually change any behaviour. It’s actually something that’s been observed in video games: a good AI tells you what it does, which makes it look smarter. One of my favourite examples of that would be the enemies in FEAR.

So by typing 'debug_ai' in the console and observing an AI empire, you can see what it has in mind:

pasted image 0 (1).png

“I don't even see the code. All I see is blonde, brunette, redhead. Hey uh, you want a drink?”​

As a simple analogy, imagine that the AI has a war minister that looks at the big picture and rates every potential target, a general staff who assign fleets to some of those objectives, and then admirals who try to lead those fleets on a tactical level to achieve those objectives.

The skulls on top of each system shows military objectives that the AI is considering (the war minister). Red ones are the ones they selected and committed some fleets to, while green ones are other options they haven’t retained for now. Finally for each individual fleet, in those task forces, you see what they are doing at present.

In our screenshot example, the AI decided that taking Tiralam was the most important objective with a score of 4500, and that they estimated that at least 11.2k fleet power was needed to accomplish this. They committed the Kilik Armada, the Jinki-Ki-Ti Armada and the the Grekki Armada to this. Since it makes little tactical sense to attack in a dispersed formation, the AI issued orders to regroup in Broon’s Singularity before proceeding on the attack (something we improved in this patch).

For convenience, the summary is also visible in the outliner:

pasted image 0 (2).png

As seen from the other side of that war​

That change alone allowed us to see where the AI was a bit weak and also made evident a few bugs in the production AI that we promptly fixed. A funny one was that in some cases a fleet would end up assigned to two different fleet groups, nicely simulating two admirals fighting over command of a fleet and issuing contradictory orders every day.

Crisis AI
The next step was to rewrite the various crisis to use the generic AI, so that any effort spent on making better would benefit all. In patch 2.6 the specific AI of the Khan, the Prethoryn, the Unbidden and the Contingency will use the same AI as the “standard” empires, with a few twists to still retain their personality.

Without spoiling every secret, here’s a few ideas:
  • The Khan doesn’t really believe in defense and will try to beat the closest systems into submission
  • The Prethoryn will swarm in every direction they can
  • The Contingency will systematically try to stop the biggest threat to the galaxy, until nothing remains
  • The Unbidden will be harder to predict, but there’s reason behind their alien way of acting.
One of the biggest challenges we faced was assigning fleets to objectives. Matching X fleets with Y out of Z objectives is not an easy task. Do we try to accomplish as many objectives as we can at the risk of spreading too thin or accomplishing nothing of value? Should we instead focus on the most valuable target and possibly end up in a big fight that we could have avoided? How often should we reconsider our options?

The current version solves this by putting a fleet power value on every target, then grabbing fleets by order of priority until it either has enough to accomplish the objective, or go over the next one. This approach showed its limits when we plugged the crisis AI into it, as it relies a lot on the size of available fleets (it doesn’t know how to split them, it can only merge them).

Teaching the AI how to split fleets proved quite interesting:

pasted image 0 (3).png

What shall we do with this knowledge?​

It took several tries to find a good balance, as the AI tended to split too much (most objectives don’t call for that much fleet power, unless you’re fighting your enemy main fleet). In the end, after trying some complex strategies such as keeping statistics on accomplished objectives and deriving a good target number from that, a simpler approach turned out more efficient: put all the nation’s offensive fleet power into one stack, and then consider splitting in 2,3 or more depending on how confident the AI feels about its military power versus its foes.

Knowing some of you like to mod our AI, here’s some new defines you may want to play with once all that hits the shelves.

Code: 
# Objective values
HORDE_INVASION_PLANNING_DEPTH = 5    # How far out does the Horde AI looks for invasion targets (in system hops)
SWARM_INVASION_PLANNING_DEPTH = 5    # How far out does the Swarm AI looks for invasion targets (in system hops)
SWARM_POP_TARGET_MULT = 1.0            # Extra target scoring for swarm (multiplied by number of edible pop on the planet)
CONTINGENCY_MEGASTRUCTURE_EXTRA_VALUE = 4    # How attractive are megastructures to the Contingency (added to the base value of 1)
UNBIDDEN_PORTAL_EXTRA_VALUE = 20            # How much does the Unbidden want to defend their portal (compared to base value of 1)
UNBIDDEN_BYPASSES_EXTRA_VALUE = 4            # How attractive are bypasses to the Unbidden (added to the base value of 1)
UNBIDDEN_RIVALS_EXTRA_VALUE = 10            # Extra target scoring for rival invaders (Aberrant and Vehement)
UNBIDDEN_TARGET_EXTRA_VALUE = 10            # Extra target scoring for randomly chosen nemesis
UNBIDDEN_PSIONIC_CONQUER_DESIRE = 20        # Extra weight added to psionic empires when rolling a nemesis (base 1 + number of owned bypasses)
UNBIDDEN_CHOSEN_ONE_CONQUER_DESIRE = 50        # Extra weight added to empire lead by the chosen one when rolling a nemesis (base 1 + number of owned bypasses)

# Fleet sizing
OFFENSE_VS_DEFENSE_STRATEGY_ALLOTMENT = 0.75 # How much of its fleet power should a country with 1.0 aggressiveness should try to commit to offensive missions
AVERAGE_FLEET_SIZE_FACTOR    = 0.05            # Ballpark estimate of the minimum size a fleet should be in relation to total fleet power
OWN_FLEET_POWER_FACTOR = 1.0                # How much does AI count its own fleet power when evaluating forces
ALLY_FLEET_POWER_FACTOR = 0.5                # How much does AI count ally fleet power when evaluating forces
ENEMY_FLEET_POWER_FACTOR = 1.0                # How much does AI count enemy fleet power when evaluating forces
FLEET_SUPERIORITY_FACTOR = 1.5                # How stronger should the AI be before it starts considering splitting fleets (fleet count = relative strength / this factor)
CRISIS_FLEET_SUPERIORITY_FACTOR = 1.0        # Same as previous but will be compared to the strongest foe in the universe

Most of those changes will be delivered in the patch coming alongside the Federations release (2.6.0), but not all. As you may imagine, changes to the military AI are quite impactful and we don’t want to release the changes without enough testing, so some of them will be delivered in the first support patch (2.6.1).

And with that, I shall leave you with @sidestep one last time.
 
Last edited by a moderator:
  • 2Like
Reactions:
Sort by date Sort by reaction score
Great work!

For everyone clamoring for better and/or more diverse AI plans, I have to imagine after a week or two with the new system the mod scene will incorporate this new system with good effects. That said, ethics driven plans are dangerous because some priorities are simply better than others. Civ 5 used a weighting system for how the AI prioritized different strategies, and these differed by country. The end results was mixed. Each AI had personality but the ones that prioritized growth, science and production had a chance to become threats and everyone else fell flat on their faces.
 
A few questions:

- If you mod in multiple crises, will they understand that their goals are mutually exclusive and try to deal with each other early on, or will they look at the others like regular empires when considering their targets or even ignore one another unless they blunder into each other's path?

- If an empire reaches the Space Age during the mid- or lategame (whether on its own or through enlightenment), will it still try to pick a mid-/lategame plan, and how badly is it hurt in that case? It'll obviously not be overly competitive since everyone has a head start, but it might be odd if a lategame arrival goes "I must get all the alloys/research/unity!" when it hasn't really solved more basic concerns.

- Similar to the above, if an empire is split into pieces due to a faction or an Ideology wargoal not enforced against the whole, will the new empire be able to handle possibly being quite far behind in some category because it only consisted of a few planets?

- Does the AI take less reliable income into account (e.g. trade deals and event modifiers) when deciding if it has achieved a certain goal, and does it understand that that income is temporary (or at least unreliable) and plan accordingly?

- Does the AI account for lowered upkeep while its fleets are docked when considering its income, or does it go "Oh no! I need to build Energy/Alloy stuff now!" every time it undocks its fleets if that drops it below the desired income and then go "I have too much Energy/Alloy stuff! Delete it!" when it docks its fleets again?

- With the military AI having been improved, is it capable of recognizing rather obvious bait if it can see a trap inside sensor range, e.g. if there's a rather weak starbase in a border system and a large fleet is waiting one hyperlane/wormhole away and rather obviously intends to pounce on anything moving into the "undefended" system (which is something the old AI didn't really consider)?

- How good is the military AI at prioritizing potentially stronger important targets over potentially weaker unimportant targets? For example, if it is approaching 100 % WE and can only reasonably expect to liberate one planet before being forced to sign a peace deal (if we assume that the enemy will do it as quickly as possible if it starts losing occupied planets), would it know to liberate a high-value planet (e.g. its capital) that might be a bit more highly defended than a weak but comparably less useful planet (e.g. a recent colony) that might be easier to reconquer but probably is the worse choice?
 
OK, but what about sector and automated-planet AI? Does it works the same, what are those plans then?
And do AI destroy/replace some buildings?
 
sidestep said:
The scoring system does not use the base building weights, they are only used when the plan system is not used. The base weight used by the plan system is calculated entirely separate from the building weights.

Defensive buildings such as fortresses is handled by another system, that's why defenses does not have a weight here :)
Click to expand...

Good to know. Is there a way to mod the system that builds fortresses? I have a modded planet decision that creates a defensive modifier, and Id love to be able to teach the AI to choose that on worlds it would want to fortify, and likewise, to want to fortify worlds that have the modifier. But might be getting off topic here since it sounds like that is handled by a different minister.

More out of curiosity then, if base building weights from the building file itself are not being used when a plan is in effect, what are the base weights of things the plan could build? Are those exposed and modable somewhere? And if we make a building that provides some benefit other than resources --for example a fancy admin building on your capital that gives a nice empire modifier of some type, but does not produce resources itself-- how do we convince the AI it is a good idea to build?
 
Y'know, once travel times became so much slower I found it wasn't about doomstack finding doomstack. I often build fortress worlds, let the AI pound against them, and hit them somewhere else. Pure doomstacking on a lot of maps will let you lose a chunk of your empire because you can have fronts separated by a year or two of travel time.
 
If there is a big difference in power, you can not declare a nemesis, so if you do not create a fleet on purpose, AI will not attack because there are not enough diplomatic minus points. Even small wars between AIs do not destroy small nations, so it will be a slow game development, so I want you to do something.
 
AI stops the planet from growing when the house is full. Humans try to increase the number of cities and increase POP as much as possible. If not, the growth of the AI empire would stop 2300 years ago. Then humans will definitely win. Nothing is as boring as a game you can win. I want the AI empire to continue growing.
 
I don't really like the AI's planet-by-planet economic planning. The AI's economy ain't even near something optimal like it can't build ecumenopolises if it can't consider a set of specialized planets planned to sustain one another and still make good left over resources in that set. For example, a set of 5 ecumenopolises; one to each specialization (alloys, consumer goods, unity, research and trade) will need, say, another set of 10 to 15 other rural specialized economy worlds where it can harvest all the commodities it needs in order to sustain the more central and strategic ecumenopolises. Of course the AI can't consider ecumunopolises and rural worlds separetely but instead it has to keep in mind a number of permutations of sets, say, ecumenopolis 1 with rural worlds 2, 3, 4 and 5; or ecumenopolis 1 and 3 with rural planets 2, 3, 4, 5, 9, 12 and 15 and so on. That would make for some serious economic planning made by the AI that could overmatch players even in the easiest dificulties.
 
So amazing! I can't wait to play with the new AI.
I hope I don't have to fight the end crises and fallen empires by my own, but get help from other nations. Currently, the other nations are sooo weak, they can do nothing against end crises and fallen empires.
 
Can we mod Plans using if statements and multipliers? Say, "mineral goal is 200, if Habitable worlds is 0.25x then goal is 50, if Difficulty is Grand Admiral then multiply goal by 4" etc?
 
Last edited:
I have two questions, both involving extreme, sudden departures from current resource production:

1. I like to play as barbarian despoilers for the challenge. If I can, I'll prefer to raid my opponent's planets for most of their pops but not claim the planets themselves unless I need to expand. In the new model, this has two effects: most AI planets will suddenly have many available jobs (and some ruined buildings), and the AI empire will suddenly be producing well below the targets on it's current plan. How will the new plan based AI react to this? My understanding is the AI will avoid adding buildings to planets that have lots of open jobs, which would be most at that point.

2. Related to the above or to the simple loss of planets during war, what happens if the AI finds itself under target on many focus goals? Will it adjust it's plan, or just prioritize based on variance between production and focus target?

In short, will an AI empire that's been drastically harmed be able to recover without stagnating?
 
Maybe im not the first here, but wouldn't it be nice if an overlord could give his vassals such plans? Best with the possibility of a long-term payment or donation. Which in turn should have an impact on relations and factions.
 
I appreciate the DD and the transparency, very nice write up. Perhaps I missed it but I'm wondering if it is possible to mod in plans that respond to competitive status?

The most important seem to be:
(1) War related e.g. desperation war (war with some exterminator/crisis type):
That highly favors alloys... with perhaps some less dramatic responses like normal war and war-prep economies.
(2) Protectorates etc:
Which may want to go with the bare minimum or alloy surplus to focus on growing their econ.
 
Will the plan start date conditions be mod able? It strikes me as a bad idea to tie economic plans to purely start dates when primitives exist.

Also, is there an ability to transition to another plan if the current one is fulfilled?
 
Plot twist, since the AI has been improved so much, crisis still aren't a threat because the AI has improved so much it can actually fend for itself (on another note, has anything happened with the Grey Tempest AI? I am very curious)
 
First off, great job implementing these changes and great job explaining them to us!

I might actually be more excited for 2.6 because of this (and the potential it unlocks for AI modders to improve things even further) than because of the diplomacy features! (Although those are really cool too!)

Some questions:

1. Are built but not-worked-in buildings treated like queued buildings? Someone else gave the example of an ecumenopolis that builds a new district and loses a ton of clerks -- would the AI feel the need to make more amenities-and-trade-value buildings to compensate, or would it go 'There are empty jobs that will meet the need, so pop growth will fix it over time'?
1A. Does the AI consider trade value a local per-planet goal or a global country-wide one?

2. What do you think about the idea of displaying a simplified version of the AI debug values, but applied to the human's empire? Perhaps a sort of optional tutorial feature for those not sure what they need to do next -- or for those whose empire is doing great but aren't sure what to do next either? (Both in terms of 'is this a good idea from a user experience perspective' and 'is it practical to implement in Stellaris')
 
I have to say, I'm loving this. Yes, it might still have issues or oversights, and there are only a few core plans. But it sets the stage for so much future potential. Also, it's very, very, appreciated to have devs sticking around in the dev diary threads to answer questions, please keep doing this! In depth Diaries and interaction afterwards are the two main pillars of having good dev diaries.

Main ways I'd like to see this improved in the future:

Coding:
  • More hooks (ability to easily check number of pops, number of systems, number of planets, etc.)
  • Specified actions/instructions (Building habitats, megastructures, ecumenopolis, terraforming)
  • Defense as a value (Building fortresses/shield generators to defend key planets. Probably get planets flagged to be defended from the miliary advisor)
  • Ways to modify a Plan (Increase the focus value of minerals by +20 if Lithoid. Decrease CG goal by 10 if above 500 in storage. Set Zro goal to 0 if Machine or Genetic Ascension path is chosen)

Scripting:
  • Different Plans for different personalities. This doesn't need to be a unique plan for every personality, but there should be a science plan for erudite explorers, a warmonger plan for genocidals/despoilers/crusaders, A pop growth for Migratory flocks and Harmonius Collectives, a unity plan for inward perfection and spiritual seekers. Just tweaking the numbers a little in various directions so that each personality feels unique in how it tends to play.
  • Specialized plans for situations. Have a "preparing for war" plan where the AI focuses more on alloys and energy. A "surrounded by enemies" plan to stockpile alloys and build up fortresses. Separate Expansion plans for if they have the space to spread out wide, or need to build up tall. A "rebuilding" plan for if they loose several planets or a large amount of pops. These could either be separate plans or they could be modifiers (see four point above).

Player Sectors:
This system would be PERFECT for a player sector. Either have options for different balances of resources, or the ability to set custom values for each. There should also be a toggle for whether the sector looks only at its own production (independent), or looks at empire level production (integrated). The former would let you set up self-sufficient sectors across your empire, the latter would fill in any gaps in your own production as you build up your planets. If added, this should be an option, so players who want to can still manage every planet.
 
You must log in or register to reply here.
Top Bottom
Back