The AI is one of every strategy game’s most important features. The bread and butter of the genre is offering difficult decisions in a replayable environment where success requires constant adaptation. Good AI enhances all three of these elements – whereas a bad one can derail the entire experience.

In the context of game AI, “good” means more than just “smart.” The job of computer opponents is more than just standing in for humans – they’re a part of the world and a gameplay system just like economics, diplomacy and warfare. Attila the Hun isn’t just there to provide a roadblock to victory – he’s there to be Attila the Hun, the frightening warlord who races in with a massive horde of horse archers, burns everything in sight, and runs off before you can muster a response. Is such a strategy productive? Hopefully! But it’s in many ways that’s only a secondary concern.

That having been said, a good AI must also put up a competent fight. This is especially true with features like combat, when the AI’s decisions are front-and-center and mistakes cannot be tucked under the rug. Such competency requires AI players to craft plans and be capable of course correcting as necessary.

For all of this to come together, you too as AI developer must have a plan. To that end, I’ve spent much of the past couple months putting together an insanely detailed breakdown for every AI subsystem and the logic each will perform. Think of it as a “first draft,” where the second will be the translation to programming code. I’ve run through repeated thought experiments to test my design and I’ve scrapped more than one critical feature when it became clear that intention and results weren’t in harmony.

I won’t be posting that work today (or ever… it’s already well over 200 pages and still growing!). But I know many of you would like to see how I envision everything fitting together, so what I will be doing is sharing my basic outline for the components that will make up an AI player. As with all of game development I have no doubt the details will change a thousand times over, but I’m fairly confident this general structure will be maintained until the game is completed, and very possibly for several future titles.

– Jon

AtG Website | Conifer Forums | ‘Like’ us on Facebook | Conifer Twitter

…

 

AI Outline

 

Info Library (IL) – Not a specific AI system, but a general name for the vast collection of processed data for an AI player… is that guy stronger than me? How dangerous it that tile? etc.

Grand Strategy AI (GSAI) – Sets the general, game-wide direction taken by the player.

Objectives AI (OAI) – Determines what the AI’s priorities are, what opportunities are available, which are worth pursuing, etc. The Objectives chosen are what shapes the decisions made by all of the following AI subsystems.

Economic AI (EA) – Allocates Resources, manages production choices and handles misc low-level economic decisions like choosing Romanization Perks.

Diplomatic AI (DAI) – Manages day-to-day diplomatic interactions, including when to offer trades, what offers we’ll accept and what specific wording is used in all exchanges.

Front AI (FAI) – Allocates Units between different opportunities, and puts in requests with EAI when more are needed.

Tactical AI (TAI) – Gives specific orders to every Unit, every turn.
 

Priority – A scale with 20 values that serves as a general measurement for how important something is. Each label can have a “+” or “-” after it to make its value slightly more or less. So a “Very Good-” Priority would be one level above a “Good+” Priority, which is itself one level above “Good”.

Zone – A small group of tiles, will probably end up being roughly 3×3.
 
 

Grand Strategy AI (GSAI)

 

Summary

The GSAI chooses a single GS, which shapes the general game-wide direction an AI player pursues. It dictates the what, but not the how – it might provide us with the goal of conquest, but it won’t tell us if we’re going to do it by sea, with mounted Units, etc. Execution details such as these are decided at the OAI level, which also takes personality traits and the current situation into account.

 

Details

In the early game, the GSAI picks a “precursor” GS, which hints the player in a direction without committing them to anything specific. Maybe we’re playing a militaristic game, but we don’t know what form that will take. Or we’re committed to being peaceful with the Romans, but we’re unsure how closely we should tie ourselves to them since we don’t yet know our proximity.

In the XML, each AI leader is assigned a fixed set of GSs it’s able to choose from. This ensures that every leader plays consistent with their personality, but doesn’t tie them down to a single strategy like “always attack everyone you see.”

 

Factors to Consider

• The AI’s intrinsic personality is by far the most important. Attila is never going to sit back and play nice with everyone, no matter what the situation is.

• Proximity to the Romans. If we’re really close we probably want to make nice. If we’re too far away then focusing heavily on them doesn’t really make sense.

• Proximity to other barbarians. If we’re completely surrounded by enemies then we probably want to lie low and play it safe until things shake out.

• Availability of resources. Should we completely lack Metal, then it doesn’t make much sense to play aggressively.

• Geography. If we’re close to the water, this opens up the possibility of naval raiding. If we’re tucked into a defensible corner, then we can afford to be a little less cautious.

• Military balance of power. If we fight a major war, lose our entire army and two-thirds of our population, then it’s probably time to change course.

 

Relationship with Other Systems

The GSAI doesn’t receive input from any other subsystem.

• OAI: The sole job of the GSAI is to alter the Priority scoring of Objectives being considered by the OAI. It makes decisions unilaterally (using data from the IL) and receives no input from anyone.

 
 

Objectives AI (OAI)

 

Summary

This is the meat of the player-level AI which sets all short and medium-term goals. Decisions are made here for everything from who to declare war on, to which resource deposits to try and grab, to when and where to migrate.

The reason why so much is done in one place is because the AI needs to have a very clear idea of how to score competing priorities. This can either be done with multiple subsystems that all “speak the same language,” or handled in a single location. We’re taking the latter approach.

However, what the OAI does not do is manage any details. If we want Metal, then the OAI figures out which Deposit is worth going after and assigns it a priority, then hands the task off to the FAI to get it done. If there aren’t enough Units available the FAI is what figures this out, and then makes a Request to the EAI.

 

Details

The OAI utilizes two different types of Objectives, lumped into a single prioritized list:

Goals – Abstract “things we want to do,” like undermining the Goths, getting more Metal, or migrating.

Opportunities – Concrete “things we can do,” like invading the Goths, harvesting the Iron Deposit on Tile X, or migrating to Region Y.

 

Objective Creation & Execution

First, every single possible Objective is analyzed for its viability, and if it’s legal, prioritized. Ongoing Objectives are processed first and given a small bias (so that the AI doesn’t change its mind all the time), followed by new Objectives. The combined list is then sorted.

Next, higher priority Objectives alter the Priority of those Objectives below them in the list – if we’ve decided we’re going to invade the Romans, it doesn’t make sense to also try to complete a Request for them!

Any Objective with at least Medium- Priority is considered Active, which means it will be acted upon by other AI systems. Everything below that threshold is completely ignored. Since every possibility is considered and scored, this is our way of “focusing the AI’s attention.”

Ultimately, the OAI has authority over all other systems. If it says to declare war, that’s what we’re going to be doing – consequences be damned.

 

Relationship with Other Systems

• GSAI: Our current GS alters the Priorities of individual Objectives within the OAI. The “Lie Low” GS will greatly reduce the Priority assigned to all “Invade Player X” Objectives.

• EAI: When the OAI wants a nonmilitary thing built, researched, chosen, etc., it puts in a Request with the EAI. Construction Requests from the OAI are extremely rare, as all Requests for anything relating to Units come from the FAI (which, in turn, derives its goals from the OAI – delegation at its finest!).

• DAI: The OAI’s Objectives shape the general approach taken by the DAI (“make trades when possible”) and even occasionally makes specific directives (“try to trade for Metal”). The OAI is completely in charge of declaring war and making peace. When negotiating peace treaties, the OAI tells the DAI the general parameters of what is acceptable. The best way to think of it is that the OAI makes the important decisions, while the DAI handles day-to-day affairs.

• FAI: The OAI provides the FAI with 2 things. First, general military goals, such as “capture Gothic Cities” or “wreak mayhem on the Huns” or “ignore everything else, protect the homeland!” The OAI also asks for specific missions, such as “capture that Hostile Stronghold” or “claim that Iron deposit.” The OAI also gets its info from the IL side of the FAI as to whether we should be fighting, and if so, how.

 
 

Economic AI (EAI)

 

Summary

The EAI balances day-to-day management of the economy with fulfilling “Requests” that come from the OAI and FAI. It’s main job is to manage the allocation of Resources, but it also handles other economic decisions such as choosing Romanization Perks.

Nearly all actions taken by the EAI will be with the goal of fulfilling Requests, but there will be times when there are no pressing matters to attend to and the EAI must maintain a healthy holding pattern.

The EAI also makes Requests of the OAI when there are circumstances that prevent it from completing Requests its received. For example, if the FAI asks the EAI to build Infantry but we have no Metal to do it with, it’s up to the OAI to rectify that (or not, if there’s more important business to attend to).

 

Details

The main task of the EAI is balancing competing Requests and the long-term health of the kingdom. If we have a Request to train Units which require Population, we’re unlikely to build them from tiny Cities that can’t really afford it, unless that really is in fact the best way to complete a very high Priority Request.

Most Requests have a target location, and the EAI factors that in when deciding where to build what. For example, if we’re planning on launching an invasion it’s better to have Units built nearby than on an island on the other side of the map.

If the score for a City/Request pairing is high enough, that Request is pushed through. If not, the Request goes away, and must be re-added the following turn by the subsystem which made it. This ensures other systems are constantly re-evaluating what they do and don’t need, and allows the EAI to be flexible and adapt to new circumstances immediately.

The EAI evaluates multiple combinations of actions to see which results in the highest score (just like basically every action taken by the TAI). This prevents situations where the AI is too “greedy” with accomplishing Requests. We’d rather the EAI accommodate two “High” Priority Requests than one that’s “Very High” and another that’s “Insignificant.”

The EAI performs exactly the same for all players in all situations. Personality biases are expressed in the Objectives chosen by the OAI.

 

Factors to Consider

• OAI economic and FAI military Requests

• Is a City a particularly good or bad fit?

  • Proximity to target location

  • Population before and after

  • Proximity to enemies

  • Other (special buildings)

• Current and projected Resource accumulation

• Current and projected maintenance (e.g. no Food during the winter)

 

Relationship with Other Systems

• OAI: The EAI receives Requests from the OAI, and can also make Requests of it. “You want us to train ships, but we don’t have any Wood!”

• DAI: The EAI tells the DAI how much we can (or cannot) afford to give up certain Resources for trade or peace treaties. However, the decision for whether or not to make the move is ultimately up to the DAI, as making peace might be a matter of life or death.

• FAI: The EAI receives construction Requests from the FAI.

 
 

Diplomatic AI (DAI)

 

Summary

The main job of the DAI is to execute on diplomatic directives provided by the OAI (which is where the important decisions like declaring war and suing for peace are made). The DAI manages day-to-day interactions of popping up when appropriate, determines the exact language in which a message is to be delivered, etc.

The DAI also looks for opportunities to complete Objectives within the parameters outlined by the OAI. There might be an Objective that really wants us to get Iron, and the Diplomatic AI looks for opportunities to trade for it. If there’s another Objective that tells us to hate on the Goths, we won’t try to trade for their Iron even it would help us.

 

Details

The DAI tests the list of actions it can perform to see if they make sense at that time, based on what Objectives have diplomatic ramifications and are in a state where action is permitted. For example, if we have an Objective to declare war and attack the Goths and the Objective says it’s ready to go, the DAI will fire away.

A particularly diplomatic AI leader will create diplomacy-specific Objectives, which the DAI then sees and reacts to.

The DAI will only attempt to make a trade if the active Objectives allow for it. e.g. if both “Get More Metal” and “Don’t Trade with Attila” are active, we’ll try to trade for Metal, but not with our Hunnic friend.

If another leader offers us a proposal, the DAI determines if it’s worth agreeing to. It analyzes each piece of the deal to see how valuable it is to us in our specific situation. For example, the value of a Resource or the cost of declaring war on someone will be tested against our Objectives.

As should be fairly clear, the goal is to look at Objectives as much as possible when deciding on what diplomatic activities to engage in.

 

Factors to Consider

• Current Objectives.

• Relations with other leaders.

• MAYBE stuff in the IL, although my preference is to use the Objectives whenever possible, as this is the part of the AI that decides what it does and does not want to do.

 

Relationship with Other Systems

• OAI: The DAI receives general diplomatic directives from the OAI via Objectives. e.g. Make nice with the Western Romans, don’t help the Huns, etc.

• EAI: The DAI can ask the EAI it how much of a Resource it’s okay to trade. However, the DAI has the final authority to make the call.

 
 

Front AI (FAI)

 

Summary

The FAI divvies Units up and ensures they’re utilized in the best way possible.

The most important job of the FAI is enacting the whims of the OAI. It might want us to take down a Hostile Stronghold, or muster an invasion, or improve an Iron Deposit. It’s up to the FAI to figure out what is required and put the pieces in motion. From there, the TAI then takes over and directs specific moves.

As part of this, the FAI organizes Units into “Fronts,” which are groups with a shared goal. An invasion force should muster at tile X and launch from tile Y. The homeland is divided into Zones A, B and C, and each needs a unique level of defense based on the threat posed to and contents of each. We need at least Z in order to safely escort a Laborer to the Horses over yonder.

 

Details

The FAI creates one Front for each:

1. Invasion or Mission requested by the OAI.

2. Zone that the player has an interest in defending.

A Front ends either when the OAI kills off the associated Objective, or when the FAI decides to abandon a Zone (on defense). Defensive Fronts are recreated every turn, since they don’t keep track of any persistent data.

Fronts that are not Defensive are classified as either a Mission, if they have a very discrete goal (such as capturing a Hostile Stronghold), or an Invasion, if its objectives are less defined (go beat up on the Huns). Collectively, these two are referred to as “I/Ms,” as they tend to be similar to one another and somewhat unlike Defensive Fronts.

All Fronts are assigned a Priority, which determines how Units are divvied up. For I/Ms, Priority is identical to the associated Objective. The Priority of a Defensive Front is based on the general defense-related Objectives in the OAI (“Defend the homeland at all costs!”) and the Value of the Zone in question (based on Assets contained, strategic usefulness, etc.).

 

Execution

The FAI assigns each I/M a Muster Tile, which is where Units will be collected in preparation for launch, and a Target Tile, which is what we’re going once the Front is finished mustering.

The OAI can assign a Front Orders which weight the score of actions taken by its Units. If a Front is essentially just a bucket of Units that have a shared goal, the Orders shape how that goal is to be achieved. Orders drive low-level behavior, such as recklessness, caution, and preventing specific Improvements from being pillaged.

A Front can be set to Withdraw if the OAI decides to cancel the Objective or abandon a Zone. A Withdrawal Target Zone is chosen, and Units belonging to that Front head towards it. Units can be pulled off of a withdrawing Front by another Front if the score is high enough. Since Defensive Fronts are recreated each turn, when one is set to Withdraw it actually creates a new I/M whose sole purpose is to get to the Withdrawal Target.

 

Unit Assignment

When deciding how many Units of what type a Front should be given, the FAI estimates the strength and makeup of enemy forces likely to be encountered. A list of minimum Values deemed necessary in each Domain (anti-Infantry, anti-Cavalry, anti-Naval, etc.) is generated.

• To determine what Units go where, each Unit/Front pairing is scored for fit, factoring in what the Front needs, the Front’s Priority, distance, general availability or scarcity of Units, etc. Once a big list has been compiled we link the Unit/Front with the highest score, and then work our way down until all Units have been assigned.

• Each time a Unit is added to a Front, the scores for all remaining Units with that Front are recalculated. Score drops by a small amount for each Unit added so that Fronts with a marginally lower Priority get Units – otherwise the big boys would never share.

• Once a Front is close to having its minimum requirements satisfied, the penalty for adding Units is reduced and then eliminated.

• If there’s not much to go around and a Front has high requirements (relative other Fronts in the area) its Priority is reduced. Generally speaking, we prefer fewer fully-staffed Fronts to several undermanned Fronts.

Based on how well or poorly each Front’s requirements are satisfied, the FAI then delivers Requests to the EAI to train new Units.

 

Special Unit Roles

Some Defensive Fronts can have part of their requirements satisfied by Reserve “Units.” These don’t actually exist, but the EAI has set Resources aside and can pop them out at any time.

I/Ms have dedicated Units for Rearguard duty. The FAI evaluates the Assets that must be guarded, the threats posed to the Front’s flank, etc. and determines how many Units it needs and where they should be stationed. I/Ms also set aside Units for Recon, which fan out to keep an eye on things, and rejoin the main force in case something important comes up.

Once the initial FAI design is functioning correctly, logic will then be added allowing Units to be loaned away by I/Ms that are still mustering to other nearby Fronts. If we’re building up a massive invasion force, this allows us to temporarily siphon off Units to take out Hostile Strongholds, escort civilians, etc.

 

Factors to Consider

• Objectives outlined by the OAI.

• Value of each Zone.

• Threat posed to each Zone.

• Current and future seasonal effects.

• Availability of Units of different Domains in a general area.

 

Relationship with Other Systems

• OAI: The FAI receives directives from the OAI via Objectives, which also determine what Orders are given to specific Fronts. The IL side of the FAI provides info to the OAI regarding our military situation.

• EAI: The FAI places Requests in to the EAI when it needs Units for various tasks.

• TAI: The FAI informs the TAI the general approach to be taken in each Zone so that it knows how to score different actions.

 
 

Tactical AI (TAI)

 

Summary

This is where individual Units are all given orders.

The TAI runs through (almost) every combination of possible moves and attacks, scores the results, and chooses the best outcome.
 

Details

A brute force approach is expensive, but necessary. It’s impossible for us to “teach” the AI how to execute a flawless tactical plan, as there are far too many factors that need to be considered.

In order to cut down on the exponential processing requirements, a series of checks will be made to limit the number of actions and combos considered. e.g. If there is no danger nearby just take the fastest path to our destination. Another way we’re speeding up execution is by placing Units into “Dependency Groups,” which separate out those which are and are not affected by one other. A Scout exploring an island has no bearing on a siege against Rome, so one need not account for the moves of the other.

Each combination of actions is assigned two scores: Safe, and Risky. A Front’s Orders (generally shaped by Objectives and leader personality) determine the modifier applied to these scores. A particularly conservative leader might need to see a significantly higher Risky score in order to go that route.

The score of an individual action is determined by a large set of rules which account for possible battle results, unseen danger, distance to destination, defensive bonuses, supply, etc. Scoring calculations are calibrating using our determination as to the Value of the Asset (Unit, Improvement, City, etc.). If moving to a tile is guaranteed to result in a Unit’s death, the negative score for that move is equal to the Unit’s Value.

Every time an attack takes place or a new enemy Unit is revealed, the Dependency Group’s action combo is re-scored. This ensures the AI is always using the most up-to-date information and won’t be ambushed.

 

Factors to Consider

• The position, number and strength of owned, friendly and enemy Units.

• Map conditions:

  • Supply available

  • Movement cost

  • Defense bonus

• Unseen Danger present on each tile:

  • General likelihood that there are hostile Units nearby, based on proximity to enemy staging points, the strength of those enemies, etc.

  • Memory of where enemy Units used to be.

  • What opponents can or might be able to see.

 

Relationship with Other Systems

• FAI: The TAI speaks with only the FAI, from which it receives the priorities it should be considering when making specific moves.