Artificial intelligence researchers at
DeepMind have mastered Pong, Chess, and Go
but some games are on another level.
The game of StarCraft II one of the most
complicated video games ever created. It
is a real-time strategy game which means
that you don't take turns you play and
you have to play fast.
It was clear that StarCraft would be a great grand
challenge for us to tackle using an
artificial intelligence, the techniques
we love namely neural networks, deep
reinforcement learning and so on so
forth.
My name is Oriol Vinyals and I'm the
lead researcher of the AlphaStar
project.
AlphaStar is an artificial intelligence system that plays the game StarCraft II.
So the first element of
StarCraft that's quite different is that
you don't get to see what's happening on
the other side of the board that is
what we call imperfect information and
is a key challenge that neither in Chess
or Go are present. Another key
difference is that the pieces of Chess
or what we call units in StarCraft are
created throughout the game, right, so you
start with very very few units and what
you're doing is deciding which pieces
are you gonna invest in creating next.
And that's very important because at the
same time you must counter pieces or
units that the other player is thinking
about without having perfect information.
The first step of AlphaStar is can you
predict where a human is gonna click
what's the most likely move so
essentially AlphaStar is gathering
information from the game where all its
units are it's getting all this data
into, not a program, but rather a
mathematical entity that is called
neural network that is able to then
process all this information.
We never hard coded what this processing means
this was fully learned from, first
imitation learning, trying to imitate
human moves and later on self play
through the AlphaStar League and after
this processing many layers of
computation of learned behavior at the top
there is essentially a single action
that will be sent back to the game, and
so this action could be create a new
unit create a worker, send this worker
to scout, move the camera to this
location and that's essentially what AlphaStar is.
Every basically 20 milliseconds in the game and every
action you see here is a result of
computation using this neural network.
By January 2019 AlphaStar was able to beat two professional players under special test conditions.
But people wanted to see if it could reach the GrandMaster level.
Beating the best players at the full game.
What's quite critical is that
humans are very creative at playing this
game so once we went online with this
agent we faced all sorts of creative
strategies that we could not possibly
have ourselves designed. So the very
first game that we entered we were
playing against obviously someone we
don't know, so it really was like a lot
of adrenaline and a great moment to let
people essentially experience what it is
to play against such a strong AI.
AlphaStar progressed by playing against
itself, against a range of human players
and crucially by playing against other
AI agents.
We have this league of agents
where all the agents are doing self play
in a manner that is perhaps a bit more
advanced than what has been done before
and there we did perhaps the most
research advances in the last few months
in which we discovered that it's not
sufficient to have only one agent that
its sole purpose is to beat itself and
its past versions of itself. It's very
important to add agents that find flaws
in the main agents right, so we call
these agents exploiter agents and if you
will it's kind of asking for a friend to
beat your current strategy and maybe
this strategy that is going to beat you
is not a great strategy in isolation but
it's certainly gonna help you become a
better player and that core idea plus
then some extra developments in
imitation learning and reinforcement
learning that we had in the project
created an artificial intelligence agent
that can play the game of StarCraft II at
a GrandMaster level.
Actually I used to play the game back
the 90s I played quite seriously the
first version of the game one of the sad
things that happened is that I kind of
waited too long to play against AlphaStar.
So the team was really like
Oriol you should play, at some point I
decided okay I'll play against the agent
I played when it was just strong enough
that there was no chance for me to win
and I just basically lost the window for
me to kind of defeat the agent sadly so
cool, victory not for me
AlphaStar has broken into the top 200
players in Europe at StarCraft II
playing the full version of the game
with all three alien races.
We are here not trying to solve
StarCraft we're trying to utilize
StarCraft for the purpose of advancing
AI research but be applied to any domain
that utilizes artificial intelligence in
any of its components and enhance for
instance its robustness its capabilities
to do imitation learning from experts
and other things that we deem super
important and that StarCraft really lets
us test on one of the most complicated
video games ever created.
Success in the real world is a little more complicated
computer games can readily provide the
data points an AI needs to master a
challenge but not every problem provides
so much accurate feedback.
The integration of AI into everyday life
will be a much longer battle
