They asked this question and
they had this knowing smile.
They said,
do you think you're going to win?
(Music)
I assumed that it had to be good.
Otherwise, they wouldn't seem
so confident.
I was pretty nervous but, I mean,
I was excited just because
of the implication, right?
- On Monday,
the bot was beatable by some players.
By Wednesday,
we played the best player in the world.
We won with him, 10-0, 12-0, maybe.
- They told me it wasn't a big deal,
because it played thousands of games
against itself,
so I shouldn't feel too bad.
But I was embarrassed.
This thing accomplished in six months what
I have set out to do for eight years.
(Music)
When I first played against AI bot,
they didn't really tell me anything
going into it.
They said, "you're going to play a 1v1."
I was pretty nervous but, I mean,
I was excited
just because of the implication, right?
If there's a bot that can play at
a super high level, then it's great.
I mean, it just pushes 'Dota' to a new
level in general.
Eight very high level players
played it and
all eight of us got stomped in game two.
People were upset, they didn't want to
give it the credit that it deserved.
Anything that can be programmed in six months
to beat eight top players one after another...
That thing was... It was unlike anything.
(Tournament host)
- You've heard about it.
Now you're going to be playing against it.
Can you comprehend the idea of a bot being
better than you?
- We wasn't sure exactly whether we'll
be able to beat Dendi on stage.
(Dendi)
- Okay, this guy is scary.
(Host)
- Does it feel like a player, like a person?
(Dendi)
- Nope.
(Host)
- He's going in.
(Dendi)
- I'm dead.
(Host)
- GG!
(Applause)
- We'd like to find
agents that are able to adapt and learn in
very new environments and accomplish
increasingly more complicated tasks.
And it turns out that
games are really great for this.
'Dota 2' is one of the most difficult games
in the world that are available right now.
- Nothing that I've ever done has quite
prepared me to play a game like 'Dota.'
(Commentator)
- They're going in once more.
Blink, looking for the Crush,
the Frostbite, perfect timing from Faith,
creating space.
- There's just so much about it, so
much you have to learn, so
much you have to grind.
It takes I'd say a year to just learn
the basics, understanding
what all the heroes and spells do.
It probably takes about, I'd say a minimum
of 12,000-ish hours to be a pro.
- Even the 1v1 version of 'Dota 2' is super complicated
The game is real time,
which means that in every second,
there are 30 decisions to be made.
If we could, we would
probably hard code all the strategies, but
it turns out that is not feasible.
We use a concept called self play.
So what we do is we start by having
an artificial neural network with initially
random parameters, and
we let it play the game.
The only influence that we have on it
is we say last-hitting of creeps is good,
dying is bad, getting damaged is bad.
What happens is that it will walk
randomly, trying out lots of things.
We can run the games usually much
faster than real time, and we can run many
copies of them in parallel.
Each of the changes of the agent makes it
slightly more efficient, slightly better.
After about two weeks, they reach
the level of professional gamers and
then they become much better than
any human players.
- It did things that we had never seen
anybody else do and it had set a type of play
style that we pretty much just copy now.
(Game audio)
When I see the bot make a play,
it clicks in my head.
I'm like, 'why aren't we doing that?'
- It's actually really fascinating to see
the strategies that they learn over time.
For example, we noticed that the bot
behaves in an extremely silly way.
In their first minute,
it started taking a lot of damage, and
we didn't know why it was happening.
It doesn't make a lot of sense.
The bot was using a baiting strategy.
If you see that your enemy is on low
health, you go very over aggressive and
you think that you're going to win.
And then the bot was defeating its
opponents within the next minute or so.
When we let it train for
a few more hours, it learned to
counter that strategy in particular and
at that point, no one was
really able to beat the bot consistently.
- No matter what happened---
like, sometimes I would outplay it...
It didn't change anything,
it just kept playing the same way.
- You cannot tilt the bot
in the way that you would humans.
It calculates what is the most optimal for
the human player to do
in the next couple of seconds.
So it's not going to be distracted by
random movements of human players or
faking attacks.
These types of strategies do not work well
against the bot.
(Game audio)
- First blood!
- I think it's going to be interesting
to see how this is going forward in the future.
We've got many requests from professional
players getting access to the bot.
It's possible that in order to be the best in the game,
maybe you'll need to use similar bots
for your regular practice.
- Because it does certain things perfectly
and you have to play perfectly
to match those things.
In that regard, you can train yourself
over and over again to just get these
pitch perfect games at a more consistent level.
So in terms of high level play,
it should make things a lot
more mechanically skilled.
(Music)
- Players do not want to reveal their
strategies to their opponents, so when they
want test out more crazy ideas, they don't
have too many people to play against.
Once we have very good 5v5 bots, you could
imagine trying out hundreds of variations
of different scenarios and trying to
improve your longer term strategies.
- I don't think that it's ever possible
for a group of five bots to ever beat
a group of five human players that are any good,
just because I don't think that there's enough
processing power in the world for
it to handle that.
The interactions between 116 or something
heroes, that go in so many different ways
that work together in different spots.
There's just too many variables that I
don't think that it can completely gather.
- In the 90s, people thought
that it's impossible to write bots
that can play chess.
And we see the similar theme
going forward over the last 10 or 20 years
and we are able to solve increasingly more
complicated environments and tasks.
I think it's definitely possible.
It's not going to be easy, but if our
research is successful, we'll be more
efficient and we'll be able to tackle more
complicated problems in the future and
solve more complicated games.
It's always impossible to tell which of
the advances turn out to be useful.
We're definitely going to see a lot more
applications of these methods in the future,
but it's too early to tell now.
(Game audio)
- Outplayed!
