do you know what's the game playing now?
this is called clash Royale and if
you've never heard of it then probably
you don't play a lot of mobile game
because it's still in the top-grossing
game of all time. Personally I've
been playing it for at least three years
and this is not me playing right now
this is my little Python script running
an AI that plays the game for me and yes
if you hadn't figured it out this video
is all about how I automated playing
clash Royale and I'm doing this because
my previous video on how I automated
playing some minigame in EVE Online was
pretty popular and after I did that with
machine learning I wanted to learn more
about machine learning and I thought I
might be able to implement something
called reinforcement learning inside
clash Royale... To be honest I never quite
got to that point because yeah it's more
difficult than I thought and I never
actually managed to implement the
machine learning algorithm inside Clash
Royale but I'm still going to talk to
you about how and why and how far I got
because I did manage to get an AI that
could farm crowns for what used to be
the clan war. I think it's become quite
useless now but it would still be
interesting as a platform to finally
implement some machine learning and I
really would like to go back to that
eventually but as you'll see there's
quite a few issues that I just haven't
quite figured out how to fix yet.
now this is I believe against the Terms
of Service of supercell so if you're
planning to do that probably don't use
your main account and, you know, realize
that supercell might not be quite happy
with what you're doing so you've been
warned it's not my fault if something
happens to your account. So if you're
planning to make a script that will play
a game for you that's the first couple
of issues that come to mind? well the
first thing about clash Royale is that
it's an Android game so you need a way
to run it on your PC if you want to run
some kind of scripts I mean there might
be a way to do it through ADB which is
the android
debugger but turns out there's a much
easier way than that called
BlueStacks. Bluestacks is an emulator for
Android that allows you to run pretty
much any Android software on your PC so
if you download it and install it it's
free and you can just log into your
Gmail account and go into the Play Store
and download clash royale and then
you're basically set to go
of course personally if you're planning
to do the AI like I did you'll probably
need to get to at least 1600 trophies
before you unlock all the cards that you
need for the deck idea I had in mind.
Basically I wanted to do a beat down
deck. If you're not familiar with clash
Royale there's a couple of different
archetype of decks and the beat down is
maybe for an AI the easiest to figure
out I thought because you can kind of
basically ignore what your enemy is
doing and just like go in with your
combo and hopes that your combo will
just annihilate the enemy where the
other archetype require a little bit
more finesse but even before we get into
the AI part I needed my script to just
be able to play the game just basically
know where the cards are how much mana
it has which screen its on which button
it has to press and this is much harder
than it looks because for example take
alpha star
and saral having not done
enough damage with this all in in a dire
situation just immediately
counter-attacking and that's as saral
sees it coming he knows that there is no
way to stop AlphaStar
It looks awesome, it's amazing it can beat the top
player at Starcraft but it also uses an
API
it doesn't need to understand the actual
pixels on the screen it's just
communicating directly with the game
engine my game doesn't do that. clash
royale doesn't have an API for machines
to learn from it so I needed a way to be
able to identify which screen I'm on
where the buttons are and a lot of other
information about what's happening in
the game at the moment some of these I
have a couple of quick tricks that I
implemented for example to know which
screen
I just find a pixel on the screen that
is a specific color for that screen for
example in the Start screen the battle
yellow button is a very specific color
at a very specific location and there's
no other screen that has this color at
this location so I just check this pixel
and if it's the if it matched the color
then I know I'm on the battle screen and
I can click the battle button and same
in the victory screen there's only in
the victory screen that you have the
crown that's yellow kind of in the
middle top part of the screen so I just
check the color here to see if it's
yellow and I know I'm in the victory
screen same thing again with all the
screen
what is nice with this is that it's
incredibly fast and incredibly
performant the disadvantage is that
every time the resolution change or that
there's an UI update that changes the
colors then I kind of have to do it all
over again and yeah it happened more
time than I would like to admit and even
to do this video I had to spend like two
days fixing my AI too because it's been
such a long time that everything had
changed so this little color trick
allowed me to figure out which screen
I'm on which allows me to start the game
the action phase then there's a lot of
stuff to figure out while I'm in the
action phase the first thing is how much
mana do I have to play my cards and for
this I use exactly the same trick I just
look at the color on different points on
the mana bar and if the red component is
high then I know it's pink and it's full
and if it's low and I know it's you know
basically empty where it's a little bit
more complicated is for the cards you
have in your hand here I have no choice
I can't use color tricks like for
everything else I had to do some kind of
pattern matching thankfully OpenCV has a
very nice and very fast way of looking
up image inside another image so I use
this with reference cutout of each of
the cards I have to figure out which one
I have in my hands the problem with that
is it's still kind of slow and when
you're running an algorithm in real time
each frame of analyzing cannot take more
than like 0.5 seconds otherwise your
reaction time will be way too slow when
you need to play a card so what I ended
up doing is that beside just using the
pattern matching of OpenCV I'm also
using numpy to cut out only the part
where I know the cards are reducing the
size of the image and reducing the size
of the image I'm looking for kind of
helps making it faster and it still
works pretty well. The final challenge in
the current implementation at least is
figuring out what units are on the board
and this one is a biggie I still haven't
gotten a perfect solution but I've
gotten much better at it my first idea
was to take the sprite sheets so I went
into the apk and I managed to extract
all the sprites that the game has but I
realized that clash royale has a lot of
sprites and it's kind of hard... like I
said pattern matching is pretty slow so
I can't just go through all the
potential sprite of all the potential
units and try to find them on the board
it's just not gonna work it's just too
slow so what I figured I would do is
that I could maybe compare a screen shot
of the empty game board with a screen
shot of the current game board to
isolate each units figuring that if I
subtract the two image what's gonna be
left is only the units but that doesn't
quite work either because it turns out
that maybe you've never noticed that the
game board is actually animated there's
sometimes like clouds or water flowing
or stuff like that
so it means that I always end up with
some garbage and it makes analyzing the
difference really hard plus if a unit
has kind of similar colors to the
background then it ends up just
disappearing altogether so yeah that
wasn't working so well and then one of
my friend noticed that every enemy unit
has its level printed in red above its
head so instead of having to pattern
match for all the units in the game I
just need to pattern match for a number
between 1 and 9 written in red and
that's surprisingly fast and it allows
me to know with very precise positioning
where every ennemy units are and right now that's what I'm
doing
it still doesn't allow me to know which
unit it is but at least I know where it
is and which level it is which can be
kind of helpful in making decision about
which card to play and where to play
them basically with just these three
little pieces of information how much
mana which card I have and how many
units are on the board I started
implementing what I call my dumb
strategy and I call it the dumb strategy
because it's actually not machine
learning it's just a bunch of if and
even as far as ifs goes it's probably
some of the worst thing I've ever done
in my life
it basically does three things; play
cards until it has giant+balloon then
once it has giant+balloon wait till it
gets ten mana and play the giant
balloon at the bridge and with this
honestly it's nearly guaranteed to get
at least one crown it will usually get
three crowned right after that but it will
take one tower down at least and that
was my objective I just wanted it to
guarantee it could take one crown before
it dies and it kind of works like that
but from this I was really hoping to get
into reinforcement learning so I started
studying how to do it and I was familiar
with scikit-learn and scipy so I
wanted to try to do it in scikit-learn
but it turns out that scikit-learn
is not really made for reinforcement
learning but I still tried to hack it into
doing reinforcement learning but yeah in
the end it never really worked and part
of it is definitely because I should
have been using something else like
tensorflow
maybe but part of it is also because I
don't think I have enough information
for good AI to emerge from the neural
network I think I need a couple more
information about the state of the board
for example I really need to know which
units are on the board not just where
they are and I also need to know how
much time there's left because when you
go into overtime you should really
prioritize taking any tower and making
sure that your tower doesn't get
destroyed but while you're in the normal
regular time it's kind of fine to lose
health on your towers so this strategy
plays a part
and I don't know how to extract the time
I would need another machine learning
maybe just to recognize the characters
of the timer and another part of the
strategy of course is knowing if you've
managed to take down the tower or how
many health there is left on your own
Tower and that's also not implemented
it's even more difficult because the
towers won't show their health until
they've taken some damage so you don't
know if the tower is healthy or
destroyed because you don't have that
information anyway I'm not quite sure
how I would do it if I could somehow
manage to get this information maybe I
would think about going back and trying
to implement the reinforcement learning
maybe learning some tensorflow but right
now it's just too much work and it's not
really that useful really let's be
honest but anyway I just wanted to show
you guys this cool AI that plays clash
Royale and I hope you liked it and also
I would really appreciate it if you hit
the like button and to subscribe because
we're getting really close to a thousand
subscriber and I just can't believe how
amazing you all are so thank you so much
and see you guys in my next video bye
