Stellaris Dev Diary #172 - Reworking the AI

[sidestep]

What is an economy AI and where do I buy one?
Hey all spacefarers, Zack aka. sidestep here. While The French Paradox has been hard at work on the military side of the AI I’ve been occupied with the AI economy, which is important as we all know that credits make the worlds go round.

So what is an economy AI and how does it work in Stellaris? Well, that could be an entire dev diary in itself and differs from game to game. But in short an economy AI decides when and how the AI should spend its resources as well as how it should work to get more of those resources. In Stellaris specifically the AI handles things such as trade, mining bases, starbase modules, policies, edicts, job favoriting and more. But perhaps most importantly the economy AI handles what is constructed on your planets!

Construction is at the core of the Stellaris economy, buildings and districts fuel your economy and help grow more pops, which in turn means more resources. Previously the economy AI did construction on a planet-by-planet basis and with no long term goal in mind. At regular intervals it would go over all planets it owned and through scripted weights it would find something “good” to build, queue it, and then move on and do the same for the next planet and so on. There exists some fundamental problems with doing construction this way however.

The Problems
Firstly it’s very script intensive and requires lots of script maintenance and balancing. If you change how a building works or basically anything in-game which changes the economic balance then you would end up having to rewrite a lot of script. “Oh, if we change this then we would need more alloys but if I increase the weighting for foundries then that would mean that this other building was not built often enough and then if we change that then…”, you see the issue?

What we ended up with was huge, complex, and very explicit and static scripts for each building with a lot of hard to follow ( and expensive to compute ) if/else statements with different weights that nobody could keep track of in the end. Below is an example AI weight script for a single building.

Spoiler: AI weight example

ai_weight = {
        weight = 0
        modifier = {
            weight = 10
            OR = {
                planet_crime < 15
                has_building = building_hall_judgment
            }
            is_capital = no
            free_amenities > 2
            owner = {
                has_monthly_income = {
                    resource = minerals
                    value > 100
                }
                has_monthly_income = {
                    resource = alloys
                    value < 100
                }
                has_monthly_income = {
                    resource = alloys
                    value > 6
                }#Check for alloy shutdown
            }
        }
        modifier = {
            weight = 200
            planet = {
                is_capital = yes
                NOR = {
                    has_building = building_foundry_2
                    has_building = building_foundry_3
                }
                num_buildings = { type = building_foundry_1 value < 3 }
                has_building_construction = no
            }
        }
        modifier = {
            weight = 400
            OR = {
                AND = {
                    owner = {
                        OR = {
                            has_monthly_income = {
                                resource = consumer_goods
                                value > 0
                            }
                            country_uses_consumer_goods = no
                        }
                    }
                    NOR = {
                        num_buildings = { type = building_foundry_1 value > 1 }
                        has_building = building_foundry_2
                        has_building = building_foundry_3
                    }
                    owner = {
                        has_ai_personality_behaviour = conqueror
                        OR = {
                            has_ai_personality_behaviour = opportunist
                            has_ai_personality_behaviour = purger
                            has_ai_personality_behaviour = propagator
                        }
                    }#Agressive empires but not robot liberators as they tend to be erudite explorers and the like.
                }#Give agressive empires 2 alloy factories on planets early on.
                NOR = {
                    has_building = building_foundry_1
                    has_building = building_foundry_2
                    has_building = building_foundry_3
                }
            }
        }
        modifier = {
            factor = 0.5
            free_jobs > 3
        }
        modifier = {
            weight = 500
            has_building = building_foundry_1
            free_building_slots = 0
        }#Repairs
    }

How does weight 200, 400 and 500 compare to other weights of buildings in other files? I have no idea.

Another problem is that an AI working this way is extremely reactive, and not very proactive. It has low income of something which leads to a script evaluating a certain building very high and it then builds that, that’s fine. But what if you’re doing quite alright? How do you then determine what to build? You would have to add even more script that says something along the line of “if you’re doing okay in all these resources then this should be built, but don’t value it too high because maybe we are in need of some auxiliary resource, hmmmm…”, you can see how this would be hard to script and would easily conflict with other script. It’s also a problem because this would all have to be manually maintained and updated as well, in each script for each building.

Yet another issue is that by doing this on a planet-by-planet basis in script we have no real idea of what has been built on any other planet or for what reason. This means that if we on one planet evaluate that an energy producing district is needed because we are running low on energy and then queue that, we will most likely queue the exact same district on the next planet we check since the situation looks exactly the same to the evaluating script even though when construction is done of the first district we might be fine and the second district would then just add to an unnecessary surplus. This might sound like a small issue but imagine if this happened on 12 different planets, it could swing an AI economy completely in the wrong direction and have massive long term consequences. The AI could even end up having to destroy some of those districts to make room for something else, making the entire effort a wasted one.

There were other smaller issues as well, but these were the main ones I wanted to address:

• Less complex and hard to maintain script
  
• More proactive AI planning
  
• No isolated planet-by-planet construction
  
• Overall better economic AI performance

The Solution aka. The Plan™
The wonderful Stellaris QA had put together a list of economic metrics that they thought the AI should be able to achieve at a certain point in time. They had a set for the early game and one for the late game. I started designing and planning how I would do this rework and came to the conclusion that the AI needed to actually plan long term, and not just react to whatever happened to them economically. This tied nicely into these performance metrics that QA had compiled so I just thought “hey, why not make these actual goals in the game!?”, and so the AI Economic Plan was born. The economic plan is at the core of the new economic AI, from the plan the AI derives exactly what it needs to reach the plan goals and decides how to do it in the long term. Plans are fully scriptable objects that simply set goals that the AI should aim for and when. Plans can either be for the early, middle or late game. They also have potential triggers and AI weights so that you can script exactly who, when and why an AI should pick a certain plan ( plans are only replaced when they are fulfilled or no longer valid ).

Spoiler: AI plan

early_default_plan = {
    type = early

    income = {
        energy = 50
        minerals = 200
        food = 50
        consumer_goods = 50
        alloys = 100
        unity = 50

        physics_research = 200
        society_research = 200
        engineering_research = 200

        exotic_gases = 1
        volatile_motes = 1
        rare_crystals = 1
        sr_living_metal = 1
        sr_zro = 1
        sr_dark_matter = 1
    }

    focus = {
        energy = 10
        minerals = 30
        alloys = 50
        food = 10
        consumer_goods = 20
    }

    pops = 500
    empire_size = 1.25

    potential = {
        country_uses_consumer_goods = yes
        country_uses_food = yes
    }

    ai_weight = {
        weight = 1
    }
}

Above is the default early economic plan used by regular empires. As you can see the potential trigger ensures that no hiveminds for example use this plan since that would mean that they started producing lots of consumer goods that they don’t need.

Below I will go through each scriptable field of the example plan, so if you’re not interested in scripting your own or more in-depth how the system works you can skip this part

Spoiler: Script Breakdown

‘type = early’
The type field determines during which part of the game this plan is considered valid, this particular one is valid for the early game so it will only be considered until you reach whatever year has been set as the mid-game start year in the game settings.

‘income = {}’
The income field contains the main economic plan goals that the AI will try to reach when having this plan. Here you can script any available strategic resource followed by how much net income the AI needs to have before it considers that resource goal fulfilled. NOTE: If a resource is not scripted here then that means that the AI will not actively try to build anything that produces that resource, it might do that because a certain building produces something else it needs though.

‘focus = {}’
The focus field can be used to set special focus goals ( net income ) for resources that the AI should specifically try to reach before anything else. This field is great to use to ensure that the AI never dips below a certain net income, and that it makes sure to produce certain resources early on even if there is no deficit of said resource. Minerals and alloys are good example of early game focus resources because the AI needs to produce a considerable amount of them to be competitive, just staying above 0 net income is not enough!

‘pops = 500’
The pops field is the goal number of pops that the AI will try to reach. It will not stop producing pops when this number is reached, however pop growth and assembly buildings will be weighted much lower in favor of more resource production when the goal has been reached.

‘empire_size = 1.25’
The empire_size field denotes, in percentage, a threshold under which the AI should try to keep its Empire Sprawl. In this case 1.25 means that the AI should try to keep its early game Empire Sprawl under 125% of capacity, so that it does not incur too many penalties but still can tolerate some.

‘potential = {}’
The potential field is a standard trigger field, scoped to the AI country, where you can script which type of countries are allowed to use this plan. The base plan potentials are just to ensure that different empire types produce only the resources they need, but you could also script things like special economic plans for when you are at war or if you are in a federation etc.

‘ai_weight = {}’
The ai_weight field is a standard weighting field that you can use to script how much the AI wants to use a certain plan. Since there is only one possible plan per country type and era right now all weights are just set to 1, but you could script up lots of different plans and weigh them differently for different empires based on for example personality, ethics or any other AI property.

Executing The Plan™
So how does the AI turn these plans into anything actionable? Good question! At regular intervals the AI will look at its current economic situation ( income, deficits, budgets etc. ) and compare it to the goals set in the currently active plan. From this it will derive exactly what it needs to reach those goals and create a build plan from that. The build plan is basically a prioritized list of buildings ( or districts ) that the AI wants to build on different planets.

This build plan can be viewed in-game while hovering an empty or locked building slot with ‘debugtooltip’ enabled and can look something like the above image ( the number is the score that the AI has calculated for that building ). The AI will recalculate this plan at regular intervals as well, so that it does not stick to old plans that are no longer relevant.

The AI constructs this build plan by finding out exactly what it can build, what those buildings actually produce when built ( including resources from jobs ) and then try to mix and match buildings to as effectively as possible fulfill the goals identified from the economic plan. This of course takes into account any building upkeep costs or tech limitations present. A benefit of this system is that you no longer have to explicitly script when the AI should build a certain building, instead it will dynamically figure out what each building produces in code. This also means that it is way quicker to add new buildings that produce different resources since you don’t have to manually script how the AI should handle them. Moreover the build plan also tries to take into account on which planet a building would produce the most and build that building there, making the AI more likely to create specialized resource-production planets like “foundry worlds”. It will also not build anything on planets that already have several free jobs. Since this is now done on an empire-wide basis and not planet-by-planet the AI can keep track of and work towards an overarching goal instead of looking at planets in relative isolation. The AI will still go through all your planets to see if any of them are missing housing, amenities or any other planet-specific resources. These buildings are however still part of the economic plan and so if you REALLY need another resource then one of your planets might have to live with an amenities deficit for a while.

So how does the AI actually prioritize the different buildings in the build plan? Another great question! To answer that we need to look at how the AI scores different buildings and districts. The scoring function takes several parameters into account: income needed to reach plan goals, income needed to reach focus goals, current income and deficits, resources produced, miscellaneous resources ( amenities, crime, housing etc. ) produced and lastly if it contributes to pop growth or assembly. The given building gets a score for each of these parameters that is factored by how much we are currently currently missing in that category and how much this building would help to rectify that, along with a defined multiplier. We then add even more score to it if it produces a resource in which we currently have a deficit or if it brings us closer to our focus goal. NOTE: The system remembers all queued buildings so if we have queued 4 mineral districts and that would fix our current mineral deficit then the 5th mineral district would not get the extra deficit score!

Spoiler: AI scoring

AI_DEFICIT_SCORE_MULT = 50                    # AI will score buildings producing resources in deficit this much more
        AI_FOCUS_SCORE_MULT = 10                    # AI will score buildings producing focus resources this much more
        AI_AMENITIES_SCORE_MULT = 2                    # AI will score amenity buildings this much more than other misc resources
        AI_HOUSING_SCORE_MULT = 5                    # AI will score housing buildings this much more than other misc resources
        AI_CRIME_REDUCTION_SCORE_MULT = 2            # AI will score crime fighting buildings this much more than other misc resource
        AI_ADMIN_CAP_SCORE_MULT = 2                    # AI will score admin cap buildings this much more than other misc resource
        AI_POPS_SCORE_MULT = 5                        # AI will score pop growth and assembly buildings this much more ( already fairly high weighted  in code )
        AI_UPGRADE_SCORE_MULT = 40                    # AI will score building upgrades this much more ( since they don't take up a new building slot )

Above you can see the multipliers applied to each of these scoring parameters, these are all defines and can therefore be modded.

​

This all sounds very complicated and “maths-y” so let’s do a quick example to illustrate. If Blorg currently has a steady income in all resources except for minerals where net income is -1, and is using the default plan as outlined above with a mineral income goal of 200 and a mineral focus goal of 30, then when the algorithm is scoring a mineral district it would…

1. See that Blorg is currently missing ~200 net mineral income to reach is goal and would give this a very high base score since the relative amount of income missing is huge
2. See that the mining district produces resources needed for a focus goal and multiply the score by a defined factor
3. See that Blorg currently has a mineral deficit and yet again multiply the score by a defined factor

The algorithm would then give a score for the build plan to use when prioritizing buildings and a mining district ( or several ) would soon be constructed by the Blorg. This would work similarly no matter which building we’re looking at, the important thing is that it scores it relatively to how much you are missing from your goal, and deducts any planned building resources from said goals. Meaning that if you’re only missing 2 mineral income the score wouldn’t be that high and if you have queued a mineral producing building then the AI would no longer look at those buildings because the ones you have planned account for all you need to reach that goal.

Evaluating The Plan™
So, does this system manage to address all the issues that I set out to? Yes, it does!

You no longer need to do complex scripts for each building, instead you can tweak the overall plan to affect how important the AI thinks a resource is. The plans also ensure that the AI is always looking forward, being proactive, instead of just reacting to deficits and certain happenings. The AI no longer looks at planets in isolation but as part of a whole, all tied together by The Plan™.

This all sounds very fancy-like and good, but now for the real question; does the economic AI perform better overall? Well...yeah!

In our tests we have seen much more robust AI economies and a better handling of the economy overall. And I’m really happy with it! Of course this wasn’t all me, I had lots of help from QA and other devs, and am very thankful for that because without them this wouldn’t have been nearly as good. Now of course, with the release of Federations closing in I would love to see what you ( the community ) manage to do with this and how you will manage to mod this into oblivion

Lastly, I of course did not forget all of the hard work you put into your mods when writing this. The old style of writing building scripts and weights still work IF there are no economic plans for the AI to use. This means that if you remove all the plans or if an AI fulfills all the plans that it can have, then it will fall back to the old system of building planet-by-planet and use the old building weights.

Thanks for reading, and good luck amongst the stars!

 

[Rodahtnov]

How will that change AI weights in jobs/buildings/related? Will we see a drastic change with Federations patch for those scripts?

(Interested in the modding side, just making AI personalities)

BTW, Amazing changes and new ways to make AI less Artificial and more Intelligent

 

[Methone]

D O G

In our screenshot example, the AI decided that taking Tiralam was the most important objective with a score of 4500, and that they estimated that at least 11.2k fleet power was needed to accomplish this. They committed the Kilik Armada, the Jinki-Ki-Ti Armada and the the Grekki Armada to this. Since it makes little tactical sense to attack in a dispersed formation, the AI issued orders to regroup in Broon’s Singularity before proceeding on the attack (something we improved in this patch).

For convenience, the summary is also visible in the outliner:

Right, this probably is what leads to the observed behavior of the AI tossing 10 5k fleets one after another into a 50k doomstack - they decided to take my low-import planet before my fleet was there, and sent everything there, and didn't think it important to stack up first.

Crisis AI

Interesting, and a good idea to give them distinct personalities, but is there anything about fixing the 'crisis is very slow and gets slower as their territory grows' issue? I had the Unbidden spawn and, despite 50 years of being unchallenged, they took 60 systems out of 1000!

Below is an example AI weight script for a single building.

Am I reading this right that the AI will never build more than 3 Foundries per world? That seems like a... not good thing.

How does weight 200, 400 and 500 compare to other weights of buildings in other files? I have no idea.

... this is, uh, not a reassuring thing to hear.

The AI constructs this build plan by finding out exactly what it can build,

What about re-building? Right now the AI never considers destroying mining districts to build cities, so it'll never build city-worlds which are, even after being nerfed to require rare resources, still the single most powerful thing in the game by a substantial margin.

Thanks for reading, and good luck amongst the stars!

Thank you for the hard work, I hope to see how this all works out!

 

[Ancillary]

@sidestep I'm curious how much weight the economic AI puts on pop growth and assembly when forming its long term plans. Right now, those factors seem to be the most important in ensuring a snowballing economy.

Also, can you share any concrete metrics for what levels of economic output the AI is able to achieve by certain years in-game?

 

[Tech Noir Synth]

Very interesting read. Thank you.

I have a couple questions regarding this. Lets start with the crisis. One of its biggest issues that prevents the crisis from ever conquering the galaxy was apparently the fact that they expand slower and slower, because most of their time is spend sending construction ships all across their territory. Did you adress this? How long does it take for each Crisis to full conquer, lets say a medium galaxy now? If the Crisis can't conquer territory, then at worst you end up in a stalemate of both the crisis and the player throwing ships at each other.

Another question about Ship Layout. Personally, me and everyone I know that plays stellaris has been playing with Starnet AI basically since Glavius AI got scrapped. Something unique about Starnet AI was that it actually teaches the AI how to build "proper" or "correct" ship types, similar to what people will use in multiplayer. Your AI might be good at managing its economy, but if they build crap ship designs, then it will simply be a waste of alloys. Was any work done on this?

About AI plans: Are there specific "builds" for empire types? For example, will a Technocracy empire focus more on science if it is not forced to invest into alloys and follow a set Ascension perk plan, just like the player would? So for example: Tech Ascendancy, Flesh is Weak (insert potential perk 3 like Arcology project), Synthethic Evolution?

With Starnet AI, I have often seen the AI go for Psionic Ascension when Spiritualist and Flesh is weak when Materialist, but so far non finished the Synth Ascension (maybe I just didn't play long enough). Will the AI do this now in the base game? How long does it take for the AI to finish one of the 3 big ascensions with both perk points and special projects like Synthethic Evolution?

And how does the AI handle purging? I'm sure you guys are aware that ever since 2.2, players have been creating huge purge planets where all pops are being purged to last many more years than what might be intended? I would like to know what your opinion is on this. Is this an intended feature? And if yes, does the AI make use of gatherin every single pop and greating huge purge-worlds to have the best economic results?

 

[Dragatus]

# Objective values
UNBIDDEN_TARGET_EXTRA_VALUE = 10            # Extra target scoring for randomly chosen nemesis
UNBIDDEN_PSIONIC_CONQUER_DESIRE = 20        # Extra weight added to psionic empires when rolling a nemesis (base 1 + number of owned bypasses)
UNBIDDEN_CHOSEN_ONE_CONQUER_DESIRE = 50        # Extra weight added to empire lead by the chosen one when rolling a nemesis (base 1 + number of owned bypasses)

So if I'm reading this right psionic ascension puts a big bullseye on your forehead when facing the interdimensional invaders. This is cool narratively, but makes the already weakest ascension path a tad worse than it already was.

I would assume bypasses means wormholes and such?

 

[adil3tr]

I have recently become minister for a young socialist republic, do you mind if I borrow some of these five year plans?

 

[Fjolsvid]

Thank you for your work and if i may a couple of questions for both of you:

Does the Crisis AI still order constructors on a node list with no checks to distance between destinations ?
Does the military AI check the strength of the opposing nation's fleets? If i have 1 million fleet power (it happens when you reach 50+ upgrades of shields, armors and guns) ships, will it ever go to war?

Does the AI actively select planet specialization or does it rely on the initial script to figure it out?

How does AI know when to colonize planets and does it plan colonization based on the information it scouted? 2.5.1 Constructs a bunch of habitats, but doesn't always colonize them, what's up with that?

 

[Levitikan]

This sounds really good, hopefully with a major improvement to there economies trading with the ai might actually be worth it and they might be a stronger threat

 

[Eelectrica]

Great Dev diaries guys. Thanks for your hard work.
Gonna be good to have an AI and end game crisis feel like a threat again.

 

[rubert]

Two questions

1. Can the military AI target enemy fleets or just systems? As a player my highest priority is enemy fleets as after they have been neutralized I can focus on other objectives.

2. Does the economy AI also plan for Megastructures and city planets? Edit: this includes stockpiling resources and waiting for the right tech before picking Ascension perks

 

[FinellyTrained]

This is depressing.

 

[AlphaAsh]

Lastly, I of course did not forget all of the hard work you put into your mods when writing this. The old style of writing building scripts and weights still work IF there are no economic plans for the AI to use. This means that if you remove all the plans or if an AI fulfills all the plans that it can have, then it will fall back to the old system of building planet-by-planet and use the old building weights.

Because you knew I'd ask

If plans are moddable, does that mean modded assets can be integrated into plans, so they don't have to wait on vanilla plans to finish before they're considered?

(Also, genius solution, if yes. This is pretty much what EDAI did, but that was one plan for all. You went and did better. Thank you.)

 

[Lord Backael]

Does the AI also factor in potential tech improvements to their economic plans? For example, if they have a mineral deficit, will they prioritize researching mining techs in addition to building more districts?

Also will the AI demolish/rebuild their districts as the game progresses? From what I've read, their general lack of an ability to do so is why they ever end up getting ecumenopoli on their own.

And finally, have habitat weights been looked at during this sweep? From my own experience, I find they spam useless trade habitats everywhere, and never fill them with more than a couple buildings. Similar point as before, I've read that they have a hard time deconstructing old orbital mining/research stations to build the more useful habitats over.

 

[Magdaki]

An interesting couple of posts. I like the first image, as it is very accurate. I'm not surprised you went with a kind of multi-agent approach towards building the overarching AI planning. This is the sensible approach as it allows for each component of the overall planning to have their own individual mechanisms. I know that generally speaking video games do not use actual AI algorithms (they're too computationally expensive or too time-consuming and cost-prohibitive to properly develop); however, I think that you could probably adapt some techniques for algorithmic multi-agent planning to the script-based pseudo-expert system that you're using.

As an example (and I'm just brainstorming here, this might be a terrible idea), but you could use a central planning server with multi-agent clients approach. The central plan component is essentially the "head of state" that comes up with an overarching desired goal. The goal is communicated to the ministers (client agents) that develop their best plan for the goal. The central planner then evaluates how well those plans should work together. Is the military planner calling for more ships, but the economic planner is calling for more consumer goods and less alloys? The central planner evaluates how well the individual plans function together, which can cause it to reevaluate its goal, or call for a new plan from a minister providing feedback; i.e., weights in your paradigm, to the client planner. So, it might tell the economic minister to try to make more alloys and tell the military minister, you cannot have so many new ships. This can be done iteratively but becomes computationally expensive. There's a lot of different ways that the original plans and revised plans can be combined. The main idea here is the individual ministers get to keep their own approaches towards solving the problem (e.g., you could rip out the script and replace with a search algorithm and everything works just fine), while giving some planning coordination throughout the network.

 

[Machvvär]

@sidestep Do sector AIs use the Plan system to handle their economy ?

 

[NHunter_rus]

@sidestep , for the mods that add additional mine-able or produce-able resources, do we need to make AI take a completely new plan once it's able to mine/produce those resources, or is it possible to "insert" a new resource as a sub-plan?

 

[Ezumiyr]

This is huge.

 

[Iosue Yu]

As someone who plays both Stellaris and Imperator:Rome, this is the best Dev Diary I have ever read. It has enough "meat". That's what I expect: How to do something in the technical perspectives and why we need it done.