  At DeepMind, we basically
work on artificial intelligence
and AI is the science
of making machines smart
I believe that this is going to end
up becoming one of the most important
technologies that humanity
will ever invent, but before I
get into that, because this session
is called The Theory of Everything
I thought I would start with a little bit
of an explanation
of my personal journey to
how I got to this point, and
why I've decided to, you
know, dedicate my life and career
to work on this problem.
  So from a young age at school, I
kind of, came to this realisation
that in some fundamental
sense there are only two subjects
really worth studying, physics
and neuroscience.
  For physics, of course, it's
all about explaining the external
world, so the external
world out there, including, of course
the entire universe, and
neuroscience and psychology
is really about conversely
explaining about what's inside
here, our internal world
  Then when I thought about this more
I actually came to the conclusion
that the mind was more important
because obviously that's the
way we actually interpret the external
world out there, and in fact, you know
'the mind interprets the world'
is something that was an idea that was
first proposed by the great philosopher
Immanuel Kant, and
  really, it's the mind that creates
our reality around us
  So this is where AI comes
in, because
  the ultimate expression of understanding
something is being able to
recreate it, and as
Richard Feynman said, one of my scientific
heroes, 'What I cannot
build, I do not truly understand,'
and that's one of the things that
I'm excited about with artificial
intelligence. I think, ultimately
it will help us understand our own
minds better.
  So my personal journey, as Darsheni
mentioned, started with games
and that's how I got into
AI. I started off playing chess
I was taught how to play chess when I
was four, ended up playing various
England chess teams, and captaining
various England junior chess teams
and by age of twelve I was a chess
master. The thing is, when you
teach a kid from a very
young age how to play chess
and if they have quite a reflective
personality like I had when I
was young, you can't help but
thinking and introspecting
about what it is about your mind
that is actually coming up with
these moves. What are the mechanisms
that allow you to make these
plans in such a complex game
as chess?
  So then when I was around eight years
old, I actually took some
winnings that I won from an international
chess tournament, and I bought my
first computer, a ZX Spectrum
48k, and I taught myself
how to programme.
  One of the first, sort of, big
programmes I can remember creating
was actually a programme to play
chess, and it didn't play very well
but it was able to beat my little brother
which I was very pleased about when I
was small.
  You know, it worked, and this was the
beginning of the path to me
towards AI.
  Now, my love of programming, and chess
and games, sort of, came together
naturally in the form of
video games, and my first
career was in creating and designing
video games. I did this for ten
years, and I wrote several bestselling
games, but probably my most famous
game I wrote was called Theme Park
  which I wrote when I was seventeen. This
is one of the first games to
use artificial intelligence
as the main gameplay component
So this game came out in the
mid-90s, in '94, and the
idea of the game was that you
designed your own Disney World
and thousands of little people
with their own desires and
characteristics came into
that Disney World and judged
and decided how much fun they had
in your theme park.
  They would go and tell their friends, and they
would come the next day.
  So this game spawned
a whole genre of games called management
simulation games, and
really, sort of, started this
whole genre of
creative games, where instead
of shooting and killing things in
games, actually you create and
built stuff yourself, and the game
would react to how
you played the game, so no two
people ended up with the same
game, because the AI adapted
to how the player played it.
  In fact all my games involved
a lot of AI,
  and then the final piece of the jigsaw
for me was, after doing this
for ten years, I sold my games company
and I went back to university
to do a PhD in neuroscience
to study how the brain itself
solved some of these hard problems
  I chose, as my topics
imagination and memory
and an area of the brain called the hippocampus
which is responsible for imagination
and memory, because these
are two of the capabilities
that we don't know how to do very
well in AI. I wanted to
see and get some inspiration for
how the brain actually solves
these problems.
  So after a couple of post-docs
at MIT and Harvard, I
then decided that I had all the
ingredients and the components
to start DeepMind, and
actually attack the AI
problem head on.
  So all these experiences then
culminated, in 2010
in me co-founding DeepMind
  and the idea behind DeepMind
was really to create a, kind
of, Apollo Programme mission
for AI.
  Now, at DeepMind, we have
over 100 research scientists
100 PhDs, top people
in their machine learning fields
and neuroscience files working
on solving AI.
  The type of AI we work on
is this neuroscience-inspired
AI, so inspired by how
the brain works at a very high
level, a systems level.
  So one way we articulate our
mission at DeepMind, it's very
easy to articulate actually, but obviously it's
hard to do, is a, kind of
two step process. So step
one, solve intelligence
and then step two, use it
to solve everything else, and
I'll come back to that right at the end
why that's not as fanciful
as it might seem.
  So more prosaically, how
is it that we're going to practically
do this? Well, we're going
to go about doing this by trying
to build the world's first general
purpose learning machine, and the
two key words here are the words
'general' and 'learning'. So
all the AI that we do
at DeepMind involves
learning algorithms. So these
are algorithms that learn, automatically
how to master tasks
from raw data. They're not
pre-programmed or handcrafted
in any way, so that's unlike most
AI out there that you've heard of
There's also the second part. We
enforce this idea of wanting
the system to be general, so
i.e. the same system or same
set of algorithms can actually operate
across a wide range of tasks
  Now, 'AI', of course, is a big buzzword
at the moment, and there have
been some big achievements in AI
that you'll of course be aware of, like
Deep Blue beating Garry Kasparov
in the '90s, and then more recently
IBM's Watson besting
the Jeopardy contestants
but in our opinion, that's still
examples of what we would call
narrow AI. So this is AI
that has been specifically
tailored or built to
tackle one problem, and one
problem only, and the hallmark
of, sort of, artificial general
intelligence, the type of AI we
work on, is that it's built to
be flexible, and general
and adaptive from the ground
up, so there's no special
casing or pre-programming
of the task involved. It has to
learn everything from first principles
  So we, sort of, think about the intelligence
problem within this framework
called 'reinforcement learning'
and it's a very simple framework to
describe, and I'm just going to describe it with this
simple diagram. So on the left-hand
side here you have the system
itself, the AI system
and the AI system finds itself
in some kind of environment
  that it's trying to achieve a goal
in, and that environment could be real
world or virtual.
  Now, the system only interacts with
the environment in two ways. So
firstly it gets observations
about the environment through its
sensory apparatus. We normally
use vision at DeepMind, but
you could use other modalities
and these observations are always
noisy and incomplete. So unlike
the game of chess, the real world
is actually very noisy
and messy, and you never have full
information about what's going on
  The job of the system is to build
the best model of the world
out there, statistical model
of the world out there based on
these noisy observations
and once it has that model of the
world, the second job of the system
is to pick the best action
that will get it closest towards its
goal from the set of actions
that are available to it at that moment
in time. Once the system has decided
which action that it, it outputs
that action, that action gets executed
it may or may not make some change
to the environment, and that drives a new
observation. This whole system
although it's very simply described
in this diagram, it has lots
and lots of hidden complexities
If we could solve everything behind
this diagram, that would be enough
for intelligence.
  We know that that's enough for intelligence
because this is the way that all mammals
including humans, learn. In humans
it's the dopamine system
in our brains that implements
reinforcement learning.
  So I'm just going to show you a few videos-
a couple of videos of the algorithm
working, but before I show you that, I just want to
explain clearly what it is that
you're going to see.
  So we used games as
a test bed for testing
the intelligence of our algorithms
and in order to have true
thinking machines or cognition
a system has to be embedded
in sensory motor data streams
sensory motor reality, and
it has to figure things out for itself
So games are actually quite a perfect
setting for this, and in fact
we used
  classic Atari games
from the '80s, which were
  designed to be challenging to humans
but are not so complex that
AI algorithms couldn't make progress
with them.
  So what I'm going to show you is the AI
playing these Atari games
but the only thing the system gets
is the raw pixels as inputs
so it's just like a human looking
at the screen, seeing all
the pixels on the screen. So there's about
30,000 numbers per
frame because the screen's 200
x 150 pixels in size
and the goal here is to simply maximise
the score. Everything is learnt from scratch
and we insist that the same system
plays all the different Atari
games, hundreds of different Atari
games.
  So I'm just going to run this video now
This is a one minute video. This is
Space Invaders, the most iconic
game, probably, on Atari
  This is the first time the AI has
ever seen this data stream, so
don't forget, it doesn't know what it's playing
it doesn't know what it's controlling, and
you can see it's actually losing its
three lives. It's controlling the rocket here
at the bottom of the screen, and it's losing
its three lives immediately, because it doesn't
know what it's doing. After you leave it playing
overnight on a single GPU
machine, you come back the next day
and now it's superhuman at the
game. It's learnt for itself
through experience, how to play
So you can see now every single
shot it fires hits something
It can't be killed anymore. It's
worked out that the pink mothership
that comes across the top of the screen
in a second is worth the most
number of points. It does these
amazingly accurate shots to do that
  Those of you who remember Space Invaders
as there are of them on the screen
they go faster. Just watch
the last shot that the rocket
does. This is predictive shot
to hit the last Space Invader
So you can see how perfectly
it, sort of, modelled
  the game world, and that data
stream. So accurately, it can
predict ahead of time what is going
to happen, just from the pixels
on the screen. So here's a second
video. It's my favourite video actually. This
is a game of Breakout. There are more
gradations here of the agent
getting better, the system getting better
So this is after 100 games
so just 100 games, and you can see
again here, the system is pretty
terrible, but you can
probably convince yourself that maybe
it's starting to get the hang of the fact
that it should move the bat towards
the ball.
  Now, this is after 300 games
so it's now hitting the ball
back pretty consistently
and it's almost never missing, so it's
about as good as the best humans
can be at this game.
  Then we thought, 'That's pretty cool. What
would happen if we just left
the machine playing the game
for a couple more hundred games?' This
amazing thing happened. What happened was
it discovered the optimal strategy
was to dig a tunnel round the left-hand
side here, and then send the ball
you know, with this unbelievable accuracy
round the back.
  So that's really cool, because actually
the brilliant programmers and researchers
who are on this programme, they're brilliant
at programming and coming up with algorithms
but they're not so good at playing Atari
  So they didn't actually know that strategy
for themselves, so this was something their
own creation taught them
  So, you know, all this work was
then actually published on the front
cover of Nature a couple of months
ago, which is the biggest science journal
in the world, and so if you're interested in reading
more about these details, you can check
it there.
  Now we're moving on to adding
things, capabilities and, like
concepts, and learning abstract
concepts, and long-term memory
  These are things that are inspired
by my work and other people's
work in neuroscience
and around mimicking
the workings of this part of the brain
called the hippocampus.
  Of course, we're not just building these algorithms
just to play Atari games
We're moving now towards 3D
games, Go, simulations
and then ultimately real robots
at some point,
  and more near-term, in terms of applications
we're using it to improve recommendation
systems like on YouTube
and also moving into predictive
healthcare applications.
  Now, I just want to end by coming back
to this, sort of, theme of Theory of Everything
So I think, you know, two of the biggest
challenges facing us
as a society are
information overload, just
the sense that there's so much
data around. You know, everyone
talks about big data, but the problem
is, what to do with it once you
have it all.
  I think loads of areas like genomics
and entertainment, you know
are all, sort of, suffering from this deluge
of data. 'How do we sift through
this data to find the insights in that
data?' Of course, personalisation
technologies are one of the technologies
that are trying to help us with that
but they don't work very well at the moment
partly because they're not very personalised
and they work by, sort of, averaging
the crowd.
  Secondly, there's the problem of system
complexity. You know, many of the systems
we would like to master as a society
like climate, disease, energy
economics, even physics
are getting so complex now
You know, it's difficult for even the
best and the smartest humans
to master it in their lifetimes
and still leave enough time for them to innovate
  So one of the reasons I work on AI
and why I think it's going to be one of the most important
technologies out there, is that solving
AI is potentially a, kind of, meta-solution
to all these other problems. We can
use it to help us solve all
of these other problems. The dream
for me, the thing I get most excited
about working on AI, is in the future
being able to make and create
AI scientists, or
AI assisted science, making
that possible, working in tandem
with human experts and human
scientists. Now, of course, there
has been a lot of news at the moment about
the ethics around AI, and
like with all new, powerful technologies
you know, it must be used ethically
and responsibly, and we're actively
researching and doing these things
and we have an ethics board that governs
the use of this technology. The technology
itself is neutral. It's always how
humans use this technology
that ends up deciding whether it's ethical
or not.
  Of course, human level AI is several
decades away, but we should start
the debate now. I just
want to end by just talking about-
we, sort of, joined forces with Google
early last year, and Google's
mission statement, of course, is to organised
the world's information, and make it universally
accessible and useful.
  One reason we decided to join forces with
Google was that we felt our mission
fitted very well with this mission
Another way of, sort of, describing
that mission is to think about empowering
people through knowledge. Another
way to think about this, kind of, AI
or artificial general intelligence
is that it's a process that automatically
converts unstructured
information into actionable
knowledge.
  So just to finish with a, sort of, slide
about the Theory of Everything.
  You know, of course, I'm a neuroscientist
as well as an AI researcher
and I think that by trying
to distil artificial intelligence
into an algorithmic construct
and comparing it to the human
mind, that might help us to unlock
some of the deepest mysteries of
the mind, like consciousness
creativity and even dreams
  Finally, I would say that, you
know, in order to find the Theory
of Everything, it might turn out
that we have to solve intelligence
first. Thanks for listening.
