In 2017's Halo Wars 2, players take command
of the UNSC Spirit of Fire once more as they
face off against a new enemy - The Banished:
a ruthless band of mercenaries led by the
Brute warlord Atriox. Stranded in deep space
with no means of help and outdated combat
technology, Captain James Cutter and his crew
must survive a war of attrition against one
of the most ruthless warriors in Covenant
history.
Developed by 343 Industries and Creative Assembly
and released for both PC and Xbox One, Halo
Wars 2 was designed from the ground up to
present Atriox and his subordinates as exciting
new additions to the Halo universe. But for
this to work in a real time strategy game,
they need to be intelligent and have a sense
of personality. I'm Tommy Thompson and in
this episode of AI and Games we're going to
explore the AI Commander system that enables
for Atriox to not only exhibit strategic intelligence,
but be capable of defeating players on various
difficulty levels *without cheating*.
I've previously discussed on the show that
sometimes developers allow AI characters and
other gameplay systems to cheat to better
facilitate the experience. As shown in episode
36, the companion AI in Ghost Recon Wildlands
are often invisible to the enemy, spot targets
for the player without seeing them and cheat
sync shots to ensure targets go down as expected.
But while companions such as the Ghost Squad
and Elizabeth Comstock exploit knowledge of
the world for their own purposes, no genre
adopts this principle more, than real-time-strategy.
RTS games notoriously cut corners to improve
the experience: from having build behaviours
that don't require them to acquire the resources
needed (hello Warcraft 3), to enemies that
ignore the fog of war and always know where
you are. One reason why some RTS games cheat
is that it's an incredibly difficult problem
to write competent AI for. You need to factor
at least two levels of intelligent decisions:
the micro scale where you're moving individual
units, and the macro scale where you're coordinating
the overall army. One need only look at my
episodes on Total War back in 2018 - where
the evolution and scale of the AI systems
involved at both micro and macro is simply
staggering. This isn't an easy problem to
solve, but Halo Wars 2 employed a fresh angle.
The designers intended for Atriox and his
commanders to exhibit strategic play without
resorting to cheating. They should make intelligent
moves on both macro and micro scale, adopt
specific play styles and adapt to various
game modes and difficulty levels. But more
critically, each AI commander should exhibit
human like behaviours: helping teach players
to play the gmae, but also inject personality
into these characters and bring them to life
within the Halo universe.
Now you might think with the recent success
of Google DeepMind training human-like StarCraft
players with AlphaStar that employing deep
learning is the way forward. But this isn't
a cost-effective and pragmatic approach for
a video game. AlphaStar isn't designed to
be an interesting opponent for varying skill
levels, its built to win at the highest levels
of competition. Plus, there's the element
of cost. As cool as it all is, AlphaStars
training process cost tens of millions of
dollars to train and that's simply impractical
for even the biggest AAA studios.
So while 343 and Creative Assembly didn't
train a machine learning AI based on human
players, they employed a similar principle.
Their AI commanders show human-like traits
in combat, because they're based on strategies
by high-ranking human RTS players, with one
specific player a critical part of the process.
Balint Marczin, more commonly known in RTS
circles as Nakamura, is a competitive RTS
player, streamer and caster and at one time
ranked as one of the top 10 players in the
world in Halo Wars for both Xbox and PC. But
in addition, Marczin is a game designer at
Creative Assembly, with his work to-date including
both Halo Wars 2 and Total War Warhammer 2.
His job was to research the macro and micro
layer strategies human players exhibit within
the game and work with the programming team
to build a system that would enable for AI
to replicate it in various scenarios.
This resulted in five macro strategies that
define a particular style of play being encoded
into the AI system:
- Rush: AI players that attempt to get a large
army on the field as quickly as possible,
destroying and capturing expansion bases as
it moves across the map.
- Boom: AI players that quickly capture multiple
bases and prepare for a war of attrition.
- Map Control: a more aggressive form of Boom
that focuses on capturing and maintaining
capture points across the map.
- Fast Tech: where the AI moves quickly to
a minimum of tier 2 units production, in an
effort to surprise their enemy with heavier
units.
- Turtle: A defensive straegy that captures
bases, stays on tier 1 units for longer, deploys
turrets and upgrades expansion bases.
Meanwhile on the micro layer there are 10
unique behaviours the AI can execute and it
allows for a lot of minute control of the
individual units. This includes common behaviours
such as focussing fire on individual units,
holding ground at a specific location or ordering
special abilities such as cloaks, shields,
grenade throws and healing. But there are
also micro tactics commonly exhibited by human
players such as the pull-back and kite. In
pullback, the AI commander will monitor a
squad of units in combat and cycle out those
that receive heavy damage from attack range,
while kiting is a tactic that keeps enemy
units just beyond attack range, allowing for
either long-range attack or opportunities
to quickly deliver damage before backing of
once more.
So at any point in time, the AI system in
game will read the current game state and
feed it into both a macro strategy planner
and a micro behaviour selector, meaning the
system can make intelligent decisions about
a long-term strategy to execute, but also
run micro behaviours to help ensure that strategy
is being maintained in the long-run. While
the micro layer is focussed purely on movement
commands, the macro layer is built into multiple
layers that focus on construction, tech ladders,
leader powers and the management of armies.
But one of the real successes of this two-tier
system, is that it allows for both macro and
micro layers to be customisable based on the
in-game scenario. In certain campaign missions
or in multiplayer skirmish matches, the AI
system has complete control of both layers,
allowing it to make its own decisions and
conduct strategy its own way. But there are
times that the game designers wanted to have
more control over the system. Campaign missions
that need more scripted behaviour, or a reduced
competency at macro behaviour in Blitz firefight,
or completing disabling the strategic elements
to become purely reactive or having zero intelligence
whatsoever like in some campaign or tutorial
missions. All of these game modes, all facilitated
by the same AI system.
So with all these systems in play, how do
you create unique personalities or customise
the AI to operate at different difficulty
levels? The trick lies in the decision making
at macro level, which is broken up into three
sub-systems: the alert manager, the strategic
decision maker and the strategy manager. The
alert manager is how the system reacts to
local information whilst making strategic
decisions, hence it can interject and influence
decisions being made to respond to dynamic
changes in the world, such as creating new
units and adding defensive structures such
as turrets. The strategic decision maker is
reliant on the strategy manager: a system
whose decisions are influenced by one or more
strategy tables.
At the beginning of a match, the strategy
manager will read in a strategy table. Each
table describes the strategy types that are
being used by this AI character and defines
their style or personality for the game. While
this defines which of the macro strategies
and micro behaviours it includes, the strategy
table loaded into memory is influenced by
the game mode and we'll talk about that in
a second. The strategy table also includes
build orders, tech and leader power unlock
orders and preferences and the type of sub-missions
this AI will frequently use - ranging from
attacking enemy structures and defensive tactics
to scouting the map and foraging for resources.
By defining all of this in a table, the game
becomes more heavily data driven. The programming
team have built all of the strategies and
behaviours for the AI to execute as well as
a XML driven format for designers to feed
strategy tables into the system. This gives
designers the freedom to develop new strategies
- and even unique executions of the same strategy
- without having to worry about the programming
side of things. This in-turn crafts the personality
and behaviour of individual leaders. This
isn't just for the Banished characters, but
also UNSC leaders that appear in Skirmish
modes such as Professor Anders, Jerome-092
and Captain Cutter himself. Plus returning
to the system overview, the strategic decision
maker extends the life of a given strategy
table, allowing the system to dynamically
add new behaviours to a chosen strategy in
order to flesh it out and show growth over
time.
Now the last thing to address is difficulty:
since a given leader will play more or less
intelligently based on the difficulty level
selected. To address that, there are two approaches:
in campaign there are multiple strategy tables
for a given leader - one for each difficulty
level - which gives level designers a lot
more control over each playthrough. While
in skirmish, a strategy table is built for
Legendary difficulty and then scaled down
based on the selected difficulty. This limits
the number of uber units, the max number of
turrets deployed, restrictions to tech and
leader power upgrades, and even limitations
on the types of micro behaviours the leader
can execute.
While this creates plenty of flexibility for
programmers and designers, this is all an
incredibly difficult balancing act to pull
off. At launch, the game had 94 strategy tables
built: 28 for campaign as well as 66 for skirmish.
These require the design team to be stay on
top of the meta of the game as more DLC and
updates roll out. Halo Wars 2 had a fairly
extensive set of DLC release post launch,
including new characters as well as an entirely
new campaign. This added another 15 campaign
tables and 111 skirmish tables bringing it
up to a total of 220. This has seen a lot
of work on design side to manage these AI
personalities after launch, with a lot of
the skirmish AIs on the lower difficulties
not just heavily tweaked but actually dumbed
down after launch based on player feedback.
This resulted in AI players who upgrades,
leader power usage and re-building of destroyed
structures much more limited as well as forcing
them to fan out a little more. Given lower-level
AI leaders would frequently turtle to a point
that it was too difficult to beat them for
more novice players.
Halo Wars 2 strove to be a polished and accessible
entry to the franchise for both seasoned RTS
players as well as those new to the genre,
with a cast of characters that evoked their
personalities when in combat against the player.
What's really interesting about this is that
the creators of the Total War franchise sought
an entirely new approach to building AI for
a strategy game, rather than adapting existing
systems from their own work; enabling programmers
and designers to work together to achieve
the best outcome for a variety of situations
and players.
Thanks for watching this episode of AI and
Games on Halo Wars 2. As we wrap up I would
like to give special thanks to Creative Assembly
- the developers of Halo Wars 2 - who provided
not only feedback but the in-engine development
footage shown throughout this episode. With
thanks in particular to Balint Marczin (Game
Designer), Dr Derek Fagan (AI Gameplay Programmer)
- whose Game AI North talk from 2017 is the
basis of the video - and Lucy Boxall (Head
of Corporate Communications) who made this
all possible. Thank you all so much for your
time, it's greatly appreciated.
Halo Wars 2 was a topic voted for by my crowdfunding
supporters who have a say in what we explore
in the series. To have a say in what is made
next, plus support me to actually make these
videos, join the AI and Games Patreon like
these lovely people already have by checking
the links on screen and in the description.
Have a good one folks, I'll be back.
