- Welcome to 2020 and welcome
to the Deep Learning lecture series.
Let's start it off today to take
a quick whirlwind tour of
all the exciting things
that happened in 17, 18 and 19 especially,
and the amazing things we're going to see
in this year in 2020.
Also as part of the series
is gonna be a few talks
from some of the top
people in learning and
artificial intelligence.
After today, of course,
start at the broad,
the celebrations from the touring award
to the limitations and the debates
and the exciting growth first.
And first of course, a
step back to the quote
I've used before, I love
it, I'll keep reusing it.
AI began not with Alan Turing or McCarthy,
but would the ancient
wish to forge the gods,
of course from Pamela
McCorduck Machines Who Think,
that visualization there
is just 3% of the neurons
in our brain of the
thalamocortical system,
that magical thing between our ears
that allows us all to see and hear
and think and reason and hope and dream
and fear, our eventual mortality.
All of that is the thing
we wish to understand.
That's the dream of
artificial intelligence
and recreate versions of it,
echoes of it, in engineering
of our intelligence systems.
That's the dream.
We should never forget
in the details I'll talk
the exciting stuff I'll talk about today.
That's sort of the reason
why this is exciting, this
mystery, that's our mind.
The modern human brain, the modern human
as we know them today
no and love them today.
It's just about 300,000 years ago
and the industrial revolution
is about 300 years ago
'cause that's 0.1% of
the developments since
the early modern human
being is when we've seen
a lot of the machinery.
The machine was born, not in stories,
but in actuality is the
machine was engineered
since the industrial
revolution and the steam engine
and the mechanized factory system
and the machining tools.
That's just 0.1% in the history
and that's the 300 years.
Now we zoom in to the 60, 70 years since
the founder, the father, arguably
of artificial intelligence, Alan Turing
and the dreams that there's
always been the dance
and artificial intelligence
between the dreams,
the mathematical foundations
and when the dreams
meet the engineering, the
practice, the reality.
So Alan Turing has spoken many times
that by the year 2000,
that he would be sure
that the Turing test, natural
language would be passed.
It seems probably he said
that wants to machine
thinking method has started,
it would not take long to
outstrip our feeble powers.
It would be able to
converse with each other
to sharpen their wits some stage.
Therefore we should have
to expect the machines
to take control.
A little shout out to self play there.
So that's the dream.
Both the father of the
mathematical foundation
of artificial intelligence and the father
of dreams in artificial intelligence.
And that dream again in the early days
was taking reality.
The practice, met with the
perception often thought
of as a single layer neural network,
but actually what's not as
much known as Frank Rosenblatt
who was also the developer.
The multilayer perception and that history
is zooming through has
amazed our civilization.
To me, one of the most inspiring things,
and this is in the world of games,
first with the great Gary Kasparov losing
to IBM deep blue in 1997
then Le Sedol losing
to AlphaGo in 2016 seminal moments
and captivating the world
through the engineering
of actual real world systems.
Robots on four wheels, as
we'll talk about today,
from Waymo to Tesla to all
of the autonomous vehicle
companies working in this space.
Robots on two legs, a
captivating the world
of what actuation, what
kind of manipulation
can be achieved.
The history of Deep Learning from 1943
the initial models from neuroscience,
thinking about neural
networks, how to model
neural networks mathematically
to the creation,
as I said, of the single layer
and the multi-layer
perceptron by Frank Rosenblatt
in 57 and 62 to the
ideas of backpropagation
and occur in neural nets in the 70s
and 80s to convolutional neural networks
and LCL is by directional RNs in the 80s
and 90s to the birth of
the deep learning term
and the new wave, the revolution
in 2006 to the image net
and Alex net, the seminal
moment that captivated
the possibility, the
imagination of the AI community,
of what neural networks
can do in the image
and natural language space
closely following years
after to the development
of the popularization
of GANs Generative Adversarial Networks.
So the AlphaGo and AlphaZero in 2016/7
and as we'll talk about language models
of transformers in 17,
18 and 19 those has been
the last few years have been dominated
by the ideas of deep learning in the space
of natural language processing.
Okay, celebrations.
This year, the Turing Award
was given for deep learning.
This is like deep learning has grown up.
We can finally start giving awards.
Yann LeCun, Geoffrey Hinton, Yoshua Bengio
received the Turing Award
for the conceptual
engineering breakthroughs
that have made deep neural networks
a critical component of computing.
I would also like to add that
perhaps the popularization
in the face of skepticism
and for those a little
bit older have known the skepticism.
Then you'll know of
service throughout the 90s
in the face of that
skepticism, continuing pushing,
believing, and working in this field
and popularizing it through in the face
of that skepticism, I
think is part of the reason
these three folks have received the award.
But of course, the
community that contributed
to deep learning is bigger, much bigger
than those three.
Many of whom might be here today at MIT,
broadly in academia, in industry.
Looking at the early key figures,
Walter Pitts and Warren McCulloch,
as I mentioned for the competition
models of the neural nets.
These ideas of that thinking that the kind
of neural networks,
biological neural networks
can have on our brain can
be modeled mathematically
and then the engineering of those models
into actual physical and
conceptual mathematical
systems by Frank Rosenblatt
57 against single
layer multilayer in 1962 you
could say Frank Rosenblatt
is the father of deep learning.
The first person to really in 62 mention
the idea of multiple hidden
layers in neural networks.
As far as I know somebody was correct me,
but in 1965 shout out to the Soviet union
and Ukraine, the person who is considered
to be the father of deep
learning, Alexey Ivankhenko
and V.G Lapa co author of that work
is the first learning
algorithms that multilayer
perceptrons multiple hidden layers.
The work on backpropagation,
not automatic differentiation.
In 1970 1979 convolution neural networks
were first introduced and
John Hartfield looking
at recurrent neural networks
now called Hotville
networks, a special kind
of recurrent neural networks.
Okay that's the early
birth of deep-learning.
I wanna mention this
because it's been a kind
of contention space now
that we can celebrate
the incredible accomplices, deep learning,
much like in reinforcement
learning and academia.
Credit assignment is a big problem
and the embodiment of that
almost a point of meme
is the great Juergen Schmidhuber.
I encouraged for people who are interested
in an amazing contribution
of the different people
in the deep learning
field to read his work
on deep learning and neural networks.
It's an overview of all the various people
who have contributed besides Yann LeCun,
Geoffrey Hinton and Yoshua Bengio.
What's a big beautiful community,
a full of great ideas
and full of great people.
My hope for this community,
given the tension
is some of you might've
seen around this kind
of credit assignment problem
is that we have more,
not on this slide, but love
that can never be enough
love in the world, but general respect,
open mindedness and collaboration
and credit sharing in the
community, less derision,
jealousy and stubbornness and silos,
academic silos within
institutions, within disciplines.
Also 2019 was the first
time it became cool
to highlight the limits of deep learning.
This is the interesting
moment in time several books,
several papers have come
out in the past couple
of years highlighting that deep learning
is not able to do the
kind of the broad spectrum
of tasks that we can think of.
The artificial intelligence
system being able
to do like re common sense
reasoning like building knowledge
bases and so on.
Rodney Brooks said by 2020,
the popular press starts having
stories that the era of
deep learning is over
and certainly there has
been echoes of that through
the press, through the Twitter sphere
and all that kind of world.
And I'd like to say that
a little skepticism,
a little criticism is really
good always for the community,
but not too much.
It's like a little spice
in the soup of progress.
Aside from that kind of a skepticism,
the growth of CVPR I clear and
Europe's all these conference
submission papers has
grown year over year.
There's been a lot of exciting research,
some will, which I'd like to cover today.
My hope in this space of deep
learning growth celebrations.
The limitations for 2020
is that there's less,
both less height unless NTI hype,
less tweets on how there's
too much hype in AI
and more solid research, less criticism
and more doing, but
again, a little criticism.
There's a little spice is
always good for the recipe.
Hybrid research, less
contentious counter productive
debates and more open
minded and interdisciplinary
collaboration across
neuroscience, cognitive science,
computer science, robotics,
mathematics, physics.
Across all of these
disciplines working together
and the research topics that
I would love to see more
contributions to as we will briefly talk
about in some domains is
reasoning, common sense
reasoning, integrating that
into the learning architecture,
active learning and lifelong learning,
multimodal multitask, learning
open domain conversation,
so expanding the success
of natural language
to dialogue, to open domain
dialogue and conversation
and then applications.
The two most exciting, one
of which we'll talk about
is medical and autonomous vehicles.
Then algorithmic ethics
in all of its forms,
fairness, privacy bias.
There's been a lot of
exciting research there.
I hope that continues.
Taking responsibility
for the flaws in our data
and the flaws in our human ethics.
And then robotics.
In terms of deep learning
application robotics.
I'd love to see a lot of development,
continued development, deeper enforcement,
learning, application robotics
and robot manipulation.
By the way, there might
be a little bit of time
for questions at the end.
If you have a really pressing question,
you can ask it along the way too.
Questions so far?
Thank God.
Okay, so first the practical,
the deep learning and deep RL frameworks.
This is really been a
year where the frameworks
have really matured and converse shores
to popular deep learning frameworks
that people have used as a
TensorFlow and PI torture.
Tessa flow 2.0 and PI torch
1.3 is the most recent version
and they've converged
towards each other taking
the best features or moving
the weaknesses from each other.
So that competition has
been really fruitful
in some sense for the
development of the community.
So on the TensorFlow
side, eager execution.
So imperative programming,
the kind of how you
would program in Python
has become the default
has been first integrated,
made easy to use and become the default.
And I'm the side towards script allowed
for now graph representation.
So do what you're used to be able to do
and what used to be the
default mode of operation
TensorFlow allow you to
have this intermediate
representation that's in graph form,
the on intensive flow
side, just the deep carious
integration and promotion
as the primary citizen,
the default citizen of the API
of the way you would Draco TensorFlow,
allowing complete
beginners just to anybody
outside of machine
learning to use TensorFlow
with just a few lines of code to train
and do inference with the model
so that that's really exciting.
They cleaned up the API,
the documentation and so on.
And of course maturing the, the JavaScript
and the browser implementation.
Intensive flow tends to
flow light being able
to run TensorFlow on, on
phones, mobile and serving.
Apparently this is
something industry cares
a lot about of course is
being able to efficiently
use models in the cloud and catching up
with TPU support and experimental versions
of PI torch mobile.
So being able to ride a
smartphone on their side,
this tense, exciting competition.
Oh, and I almost forgot to mention
we have to say goodbye to
our favorite Python two.
This is the year that support finally
and the January 1st, 2020
support for Python two
and TensorFlows and PI tours support
for Python two is ended.
So goodbye print goodbye CRO world.
Okay, on the reinforcement learning front,
we're kind of in the same space
as a Java script libraries are in.
There's no clear winners coming out
if you're a beginner in the space.
The one I recommend is
a, as a fork of OpenAir
baselines is stable
baselines, but there's a lot
of exciting ones.
Some of them are really
close to built on TensorFlow.
Some are built on PI torch.
Of course from Google, from
Facebook, from a deep mind.
Dopamine TFA agents tends
to force most of these
I've used, if you have specific questions
I can answer them.
So stable baselines is
the open a base on his for
cause I said these
implements a lot of the basic
deep RL algorithms PPO
way to see everything good
documentation and just
allows very simple minimal
few lines of code
implementation of the basic
the matching of the basic algorithms
of the open air gym environments.
That's the one I recommend.
Okay, for the framework
world, my hope for 2020
is framework agnostic research.
So one of the things that
I mentioned is PI torch
has really become almost
overtaking TensorFlow
in popularity in the research world.
What I would love to see
is being able to develop
an architecture in TensorFlow or develop
an and PI torch, which you currently can
and then a trend once you train the model
to be able to easily
transfer it to the other.
From Picador, she tends to
flow from TensorFlow to PI
torch currently takes
three, four, or five hours.
If you know what you're doing
in both languages to do that.
It'd be nice if, if
there was a very easy way
to do that transfer, then the maturing
of the deep RL frameworks,
I'd love it to see open AI,
step up the mind to
step up and really take
some of these frameworks to maturity
that we can all agree on.
A much like open AI gym
for the environment world
has done and continued work that Charisse
has started and many other
rappers around TensorFlow
started a greater and
greater abstractions allowing
machine learning to be
used by people outside
of the machine learning field.
I think the powerful thing
about supervise, sort of
basic vanilla supervised
learning is that people
in biology and chemistry
in neuroscience in in
physics, in astronomy
can deal with a huge amount of data
that they're working with.
And without needing to
learn any of the details
of even Python.
So that, that I would love to see greater
and greater abstractions
which empower scientists
outside the field.
Okay natural language processing.
2017, 2018 it wasn't, the
transformer was developed
and it's power was demonstrated
most, especially by Bert.
Achieving a lot of
state of the art results
and a lot of language benchmarks
from sentence classification to tagging,
question answering and so on.
There's hundreds of data sets
and benchmarks that emerge.
Most of which Bert has dominated in 2018,
2019 was sort of the
year that the transformer
really exploded in terms of
all the different variations.
Again, starting from Bert
XL net it's very cool
to use Bert in the name
of your new derivative
of a transformer, a Roberta distill bird
from pugging face Salesforce open AI's GPT
to of course Albert and
Megatron from Nvidia.
Huge transformer.
A few tools have emerged.
So one on hugging face is a company
and also a repository that has implemented
in both PI torsion TensorFlow
or a lot of these transformer
based national language models.
So that's really exciting.
So most people here
can just use it easily.
So those are already pre-trained models.
And the other exciting stuff is the patch.
Sebastian ruder, great researcher
in the field of natural
language processing
has put together an LP progress,
which is all the different
benchmarks for all the different
natural language tasks tracking
who sort of lead a boards
of who's winning where.
Okay I'll mention a few
models that stand out.
The work from this year,
Megatron LM from Nvidia
is basically taking, I
believe the GPT to transform
a model and just putting
it on steroids, right?
8.3 versus 1.5 billion parameters.
And a lot of interesting stuff there
as you would expect from Nvidia.
Of course there's always
brilliant research
but also interesting
aspects about how to train
in a parallel way model
and data parallelism in the training.
The first breakthrough results
in terms of performance,
the model that replaced
Bert as King of transformers
is XL net from CMU and Google research.
They combined the BI
directionality from Bert
and the the recurrence
aspects of tress home XL,
their relative positional embeddings
and the recurrence mechanism
of transformer Excel
to taking the bide directionality
and the recurrence.
Combining into chief state
of the art performance
on 20 task.
Albert is a recent addition
from Google research
and it reduces significantly the amount
of parameters versus
Birch by doing a parameter
sharing across the layers
and it has achieved state
of the art results on
12 NLP tasks including
the difficult Stanford
question answering benchmark
of squad two and they provide the provided
open source TensorFlow
implementation including
a number of raid to use
pre-trained language models.
Okay, another way for
people who are completely
new to this field, a bunch of apps, right?
With transformers, one
of them from hugging face
popped up that allows you
to explore the capabilities
of these language models
and I think they're quite
fascinating from a
philosophical point of view.
And this, this has
actually been at the core
of a lot of the tension of
how much do these transformers
actually understand basically memorizing
the statistics of the
language in a self supervised
way by reading a lot of texts.
Is that really understanding?
A lot of people say no
until it impressed us
and then everybody will
say it's obvious but right.
What transformer is a really
powerful way to generate
texts to reveal to you
how much these models
really learned before
this yesterday actually
just came up with a bunch of prompts
on the left is a prompt.
You give it the meaning
of life here for example,
is not what I think it is.
It's what I do to make it.
And you can do a lot of prompts
with this nature's very profound.
And some of them will be just absurd.
You'll make sense of it statistically,
but it would be absurd
in reveal that the model
really doesn't understand the fundamentals
of the prompt as being provided.
But at the same time
it's incredible what kind
of text is able to generate.
Okay the limits of deep learning.
I was just having fun
with this at this point.
It's still the, are still in the process
of being figured out very true.
Had to take this most important person
in the history of deep
learning is probably Andrew
and I have to agree.
So this model knows what it's doing.
And I tried to get it
to say something nice
about me and that's a lot of attempts,
so this is kind of funny is finally did
it did one I said
Let's frame his best
qualities that he's smart
said finally, but it's never
nothing but ever happens,
but I think he gets more
attention than ever.
Every Twitter comment
ever and it's very true.
Okay a nice way to sort of reveal
through this that the
models are not able to do
any kind of understanding of language
is just to do prompts
that show understanding
of concepts, of being able to reason
with those concepts,
common sense reasoning.
A trivial one is doing two
plus two is a three five
is a six seven.
The result is simply equation
four and two plus three
is like you got it right and
then it changed his mind.
Okay, two minus two is seven so on.
You can reveal any kind of reasoning,
you can do a blocks, you
can ask it about gravity,
all those kinds of things.
It shows that it doesn't
understand the fundamentals
of the concepts that are
being reasoned about.
And I'll mention of work
that takes it beyond
towards that reasoning world
in the next few slides.
But I should also mention
with this GPT to model,
if you remember about a year ago,
there was a lot thinking
about this 1.5 billion
parameter model from open AI.
It is so the thought was
it might be so powerful
that it would be dangerous.
And so the idea from opening eyes
when you have an AI
system that you are about
to release that might
turn out to be dangerous
in this case used probably by Russians,
fake news for misinformation
that's the kind
of thinking is how do we release it.
And I think while it
turned out that the GPT
two model is not quite so dangerous,
the humans are in fact more dangerous
than AI currently.
That thought experiment
is very interesting.
They released a report,
unreleased strategies
and the social impacts of language models
that almost didn't get as much intention
as I think it should.
And it was a little bit disappointing
to me how little people are worried
about this kind of situation.
There was, it was more
of an eye roll about,
Oh, these language models aren't as smart
as we thought they might be.
But the reality is once they
are, it's very interesting
thought experiment of
how should the process
go of companies and experts communicating
with each other during that release.
The support think things
through some of those details.
My takeaway from just reading the report
from this whole year of that
event is that conversation
on this topic are difficult
because we as the public
seem to penalize anybody trying
to have that conversation.
And the model of sharing
privately confidentially
between ML machine learning organizations
and experts is not there.
There's no incentive or a model
or a history or a culture of sharing.
Okay, best paper from ACL,
the, the main conference
for languages was on the difficult task,
so we've talked about language models.
Now there's the task
taking it a step further
of dialogue, multi-domain
task oriented dialogue.
That's sort of like the next challenge
for dialogue systems.
And they've had a few ideas
on how to perform dialogues,
stay tracking across domains achieving
state of the art
performance on multi walls.
It was just a five domain challenging,
very difficult fi domain,
a human to human dialogue dataset.
There's a few ideas there.
I should probably hurry up
and start skipping stuff.
On the common sense reasoning
which is really interesting
this one of the open questions
for the deep learning
community at community
in general is how can
we have hybrid systems
of whether it's symbolic AI, deep learning
or generally common sense
reasoning with learning systems.
And there's been a few
papers in this space.
One of my favorite, some
Salesforce on a building,
a dataset where we can start
to a do question answering
and figuring out the concepts
that are being explored
in the question and
answering here the question
while eating a hamburger with friends,
what are people trying
to do multiple choice,
have fun, tasty and digestion.
The idea that needs to be generated there
and that's where the
language model would come in.
Is that usually a hamburger with friends?
Indicates a good time.
So you basically take the question,
generate the common sense concept,
and from that be able to
determine the multiple choice,
what's being, what's happening,
what's the state of affairs
in this particular question.
Okay, Alexa prize again
hasn't been received
nearly enough attention to that.
I think it should have
perhaps because there
hasn't been major
breakthroughs, but it's a open
domain conversations that all of us,
anybody who owns an Alexa
can, can participate
in as a a provider of data.
But there's been a lot of amazing work
from universities across
the world on the Alexa
prize in the last couple of years
and there's been a lot of
interesting lessons summarized
in papers and blog posts.
A few lessons from Alquist
team that I particularly like.
And this is kind of echoes
the work in IBM Watson.
Well, the jeopardy challenge
is that one of the big ones
is that machine learning
is not an essential tool
for effective conversation yet.
So machine learning is
useful for general chit chat.
When you fail at deep
meaningful conversation
or actually understanding what the topic
we're talking about.
So throwing in chitchat
and classification,
sort of classifying intent,
finding the entities,
detecting the sentiment of the sentences.
That's sort of a an assistive tool,
but the fundamentals of the conversation
are the following.
So first you have to break it apart.
Sort of conversation is a, you can think
of it as a long dance and the way
you have fun dancing is you break
it up into a set of moves and turns
and so on and focus on that sort of live
in the moment kind of thing.
So focus on small parts
of the conversation
taken at a time and then also have a graph
sort of conversation is
also all about tangents.
So I have a graph of topics
and be ready to jump
context from one context
to the other.
And back, if you look at
somebody who's natural
language conversations that they publish,
it's just all over the
place in terms of topics.
You jump back and forth.
And that's the beauty, the humor, the wit,
the fun of conversations.
You jump, jump around from
topic to topic and opinions.
One of the things that
natural language systems
don't seem to have much is opinions.
If I learned anything,
one of the simplest way
to convey intelligence,
it's to be very opinionated
about something and confident.
And that's, that's a
really interesting concept
about constantly and in general
there's just a lot of lessons.
Oh, and finally, of course,
maximize entertainment, not information.
This is true for autonomous vehicles.
This is true for natural
language conversation is fun
should be part of the objective function.
Okay, lots of lessons to learn there.
This is really the Lubner prize,
the Turing test of our generation.
That's, I'm excited to see
if there's anybody's able
to solve the Alexa prize.
Again Alexa Prize is your
task would talking to a bot.
And the measure of quality is the same
as in a lot of enterprises
just measuring how good
was that conversation.
But also the task is to try to continue
the conversation for 20 minutes.
If you try to talk to a bot today like
and you have a choice to talk to a bot
or go do something else, watch Netflix,
the you last probably
less than 10 seconds,
you'd be bored.
The point is to continue trapping you
in the conversation
because you're enjoying
it so much.
And the 20 minutes is, that's
a really nice benchmark.
Four passing the spirit
of what the Tory tested
for examples here from the Alexa prize
and the was bought.
So the difference in the
two kinds of conversations.
So Alquist says, have you been in Brazil?
The user says, what is
the population of Brazil?
Alco says it is about 20
million users says, well, okay.
This is what happens a
lot with like I mentioned,
multi-domain conversation is once you jump
to a new domain, you stay there.
Once you've switched
contexts, you stay there.
The reality is you want
to jump back and continue
jumping around like in the second
most more successful conversation.
Have you been in Brazil?
What is the population of Brazil?
It is around 20 million.
Anyway, I was saying,
have you been in Brazil?
So jumping back in context,
that's how conversation goes.
Tangent to tangent and back.
Quickly, there's been a
lot of sequence to sequence
kind of work using natural language.
To summarize a lot of applications.
One for me I cleared that
I wanted to highlight
for from Technion that I
find particularly interesting
is the abstract syntax, tree
based summarization of code.
So I'm modeling computer code, this case,
sadly Java and C sharp in
in trees, in syntax trees
and then using operating on those trees
to then do the summarization in text here.
An example of a basic
power have to function
on the bottom right in Java.
The code two sec summarization
says get power of two.
That's an exciting
possibility of automated
documentation of source code.
I thought it was particularly
interesting in the future.
There's was bright, okay.
Hopes for 2020 for natural
language processing is reasoning.
Common sense reasoning becomes greater
and greater part of the
transformer type language
model work that we've seen
in the deep learning world.
Extending the context from thousands,
from hundreds of thousands of words
to tens of thousands of words.
Being able to read entire
stories and maintain
the context which transformers
again with XL net fast
Homer Excel is starting to
be able to do but we're still
far away from that longterm
lifelong maintenance
of context, dialogue, open
domain dialogue forever.
Since Alan Turing to today is the dream
of artificial intelligence
being able to pass
the Turing test and the dream of a sort
of natural language model
transformers are self supervised learning
and the dream of Yann
LeCun, is to for these kinds
of what previously were
called unsupervised,
but these calling now self supervised
learning systems to be
able to sort of watch
YouTube videos and from
that started to form
representation based on
which you can understand
the world sort of the, the hope for 2020
and beyond is to be able to transfer
some of the success of transformers
to the world of visual information.
The world of video for
example, deep RL and self play.
This has been an exciting year,
continues to be an exciting
time for reinforcement
learning in games and robotics.
So first Dota two an open
AI, an exceptionally popular
competitive game, e-sports
game that people compete
when millions of dollars with.
So this, this is a lot of
world-class professional players.
And so in 2018 open AI
five, this is a team play
tried their best at the international
and lost and said that
we're looking forward
to pushing five to the next level,
which they did in April, 2018 they beat
the 2018 world champions
in five on five play.
So the key there was compute
eight times more training
compute because the actual
compute was already maxed out.
The way they achieve
the eight X is in time
simply training for longer.
So the current version
of OpenAI five is Jacob,
we'll talk about next Friday
has consumed 800 pedo flop
a second days and experienced
about 45,000 years
of Dota self play over 10 real time months
again behind a lot of
the game systems talk
about they use self place so
they play against each other.
This is one of the most exciting concepts
in deep learning systems that
learn by playing each other
and incrementally improving in time.
So starting from being
terrible and getting better
and better and better and
better and you're always
being challenged by a
slightly better opponent
because of the national
process of self play,
that's a fascinating process.
The 2019 version, the last
version of open AI five
it has a 99.9 win rate
versus the 2018 version.
Okay, then deep mind also
in parallel has been working
and using self play to solve
some of these multi-agent
games, which is a really
difficult space when people
have to collaborate as
part of the competition.
That's exceptionally difficult
from the reinforcement
learning perspective.
So this is from raw
pixels, solve the arena,
capture the flag game, quake three arena.
One of the things I love,
just as a sort of side note
about both opening eyes and deep mind
and general research and
reinforcement learning.
There will always be one or
two paragraphs of philosophy
in this case from deep mind.
Billions of people inhabit the planet,
each with their own
individual goals and actions,
but still capable of coming
together through teams,
organizations and societies,
and impressive displays
of collective intelligence.
This is a setting we
call multiagent learning.
Many individual agents
must act independently,
yet learn to interact and cooperate.
Well, the agent, this is
immensely difficult problem
because with co adapting agents,
the world is constantly changing.
The fact that we seven
billion people on earth,
people in this room,
in families in villages
can collaborate while
being for the most part
self interested agents is fascinating.
One of my hopes actually
for 2020 is to explore
social behaviors that
emerge in reinforcement
learning agents and how those
are echoed in real human
to humans social systems.
Okay, here's some visualizations.
The agents automatically figure out,
as you see in other games,
they figure out the concepts.
So knowing very little,
knowing nothing about the rules
of the game, about the
concepts of the game,
about the strategy and the behaviors
they are able to figure it out.
There's the TST visualizations
of the different States,
important States and concepts in the game
that they figures out and
so on. Skipping ahead,
automatic discovery of
different behaviors.
This happens in all the different games
and talk about from Dota to StarCraft,
to quake the different strategies
that it doesn't know about.
It figures out automatically
and the really exciting work
in terms of the multi-agent
RL on the deep mind side
was the beating world-class players
and achieving grand master level and game.
I do know about, which is StarCraft.
In December, 2018 Alfa
started beating mana,
one of the world's strongest
professional soccer players,
but that was in a very
constrained environment
and it was a single
race, I think a protest
and in October, 2019 off
of star beach Grandmaster
level by doing what we humans do.
So using a camera, observing
the game and playing
as part of against other humans.
So this is not an artificial sized system.
This is doing exact same process.
Humans would undertake
an achieved grand master,
which is the highest level.
Okay, great.
I encourage you to observe
a lot of the interesting
on their blog posts and
videos of the different
strategies that the there are RL agents
able to figure out.
Here's a quote from the
one of the professional
StarCraft players, and we
see this with alpha zero two.
And chess is alpha stars
and intriguing unorthodox
player one with the reflexes
and speed of the best pros,
but strategies and style
they're entirely zone.
The way alpha star was
trained with agents competing
against each other in a league
has resulted in gameplay.
That's unimaginably unusual.
It really makes you
question how much the stock
has diverse possibilities.
Pro players have really explored
and that's the really exciting
thing about reinforcement
learning agent in chess and go and games
and hopefully simulated
systems in the future
that teach us, teach experts
to think they understand
the dynamics of a particular
game, a particular
simulation of new strategies,
of new behaviors to study.
That's one of the exciting applications
from almost a psychology perspective.
I'd love to see reinforcement
learning push towards
and on the imperfect information games
side poker in 2018, CMU no Brown.
I was able to beat a head to head to head,
no limit Texas, hold them.
And now team six player, no limit, Texas,
hold them against professional players.
Many of the same results.
Many of the same approaches
was self play iterative
Monte Carlo and there's a bunch of ideas
in terms of the abstractions.
So there's so many possibilities
under the imperfect
information that you
have to form these bins
of abstractions in both
the action space in order
to reduce the action space
and the information abstraction space.
So the probabilities of
all the different hands
that can possibly have and
all the different hands
that the betting strategies
could possibly represent
and sort of you have to
do this kind of course
planning so that they
use self play to generate
a course blueprint
strategy that in real time
they then use Monte caller
search to adjust as they play.
Again, unlike the deep mind open,
I approach very few, very
minimal compute required
and they're able to
achieve to beat to beat
world-class players.
Again, I like this is getting quotes
from the professional players
after they get beaten,
so Chris Ferguson, famous worlds,
he's a poker player, a said pluribus.
That's the name of the agent
is a very hard opponent
to play against.
It's really hard to pin him
down on any kind of hand.
He's also very good at making
then value bets on the river.
He's very good at extracting
value out of his good hands,
sort of making bets without
scaring off the opponent.
Darren Aliya said its major strength
is its ability to use mixed strategies.
That's the same thing
that humans try to do.
It's a matter of execution for humans
to do this in a perfectly random way
and to do so consistently.
Most people just can't.
Then in the robotic space has been a lot
of applications of reinforcement learning.
One of the most exciting
is the manipulation,
sufficient manipulation
to be able to solve
the Rubik's cube.
Again, this is learned through
reinforcement learning.
Again because self plays in
this context is not possible.
They use automated domain
rent randomization,
ADR, so they generate
progressively more difficult
environments for the hand.
There's a giraffe head there,
you see there's a lot of perturbations
to the system so they mess with it a lot
and then a lot of noise
injected into the system
to be able to teach the hand
to manipulate the cube
in order to then solve
the actual solution of figuring
out how to go from this particular face
to the solve cube is an obvious problem.
The, the, this paper
in this work is focused
on the, the much more difficult learning
to manipulate the cube.
It's really exciting.
Again, a little philosophy
as you would expect
from open AI is they have this idea
of emergent metal learning.
This idea that the capacity
of the neural network
that's learning this
manipulation is constraint.
While the ADR, the automatic
domain randomization
is progressively harder
and harder environment.
So the capacity of the environment
to be difficult is unconstrained.
And because of that there's a, an emergent
self optimization of the neural network
to learn general concepts as opposed
to memorize particular manipulations.
The hope for me in a deeper
enforcement learning space,
I aim for 2020 is the continued
application of robotics,
even sort of a legged robotics
but also robotic
manipulation, human behavior.
So use of multi-agent
self plays I've mentioned
to explore naturally
emerging social behaviors,
constructing simulations
of social behavior
and seeing what kind of
multi human behavior emerges
in self play context.
I think that's one of the
nice, there are always,
I hope there'll be like
a reinforcement learning
self place psychology department one day.
Like where you use
reinforcement learning to study,
to reverse engineer human behavior
and study it through that way.
And again, in games, I'm not
sure what the big challenges
that it remain, but I would love to see,
to me at least, it's exciting
to see learned solution
to games, to self play
science of deep learning.
I would say there's been
a lot of really exciting
developments here that
deserve their own lecture.
I'll mention just a few here
from MIT and really 2018
but it sparked a lot of interest in 2019
and follow on work is the idea
of the lottery ticket hypothesis.
So this work showed that sub networks,
small sub networks
within the larger network
are the ones that are
doing all the thinking.
The same results in
accuracy can be achieved
from a small sub network
from within annual network
and they have a very
simple process of arriving
at a sub network of randomly
initializing in your network.
That's I guess the lottery
ticket train the network
until the converges.
This is an iterative
process, proven the fraction
of the network with low weights a reset,
the waste of the remaining network
with the original initialization.
He's same lottery ticket
and then train again
the pre the pruned untrained
network and continue this
iteratively continuously
to arrive at a network
that's much smaller using the
same original initializations.
This is fascinating that
within these big networks
there's often a much smaller network
that can achieve the
same kind of accuracy.
Now, practically speaking,
it's unclear what that,
what are the big takeaways there except
the inspiring takeaway that
there exist architectures
that are much more efficient.
So there's value in investing
time in finding such networks.
Then there is this
intake of representations
which again deserves its own lecture.
But here showing 'em a
10 vector representation
and the goal is where each
part of the vector can learn.
One particular concept about a dataset.
Sort of the dream of unsupervised learning
is you can learn
compressed representations
where everyone thing is disentangled
and you can learn some fundamental concept
about the underlying data that can carry
from data set the data set to data set.
They said that's
disentangle representation.
There's theoretical work best.
I see them on paper in 2019
showing that that's impossible.
The, so disentangled
representations are impossible
without some without inductive biases.
And so the suggestion
there is that the biases
that you use should be made
explicit as much as possible.
The open problem is finding
good inductive biases,
fond supervise model selection that work
across multiple data set that
we're actually interested
in a lot more papers.
But one of the exciting
is the double descent idea
that's been extended and
to the deep neural network
context by open AI to
explore that the phenomena
that as we increase the
number of parameters
in neural network, that test
error initially decreases
increases and just as the model is able
to fit the training set
undergoes a second descent.
So decrease, increase, decrease.
So there's this critical moment of time
when the training set
is just fit perfectly.
Okay and this is the opening I shows
that it's applicable
not just the model size
but also the training
time and data set time.
This is more like an open problem of why
this is trying to understand this
and how to leverage it in
optimizing training dynamics
and neural networks.
That's a, there's a lot
of really interesting
theoretical questions there.
So my hope there for the
science of deep learning
in 2020 is to continue
exploring the fundamentals
of model selection, train dynamics.
And the folks focus on the performance
of the training in terms of
memory and speed is walked on
and the representation characteristics
with respect to architecture
characteristics.
So a lot of the fundamental work there
and the understanding,
neural networks two areas
that I had told to sections
on and papers, which is super exciting.
My first love is graphs.
So graph neural networks
as a really exciting
area of deep, deep learning,
a graph convolution
neural networks as well for solve,
solving combinatorial problems
and recommendation systems
that are really useful
in any kind of problem
that is fundamentally can
be modeled as a graph.
It can be then a solved
or at least aided in.
And you'll notice there's
a lot of exciting area
there and basion deep learning
using patient neural networks.
That's has been for several years,
an exciting possibility.
It's very difficult to train
large Beijing networks,
but in the context that you can,
and it's useful small
datasets, providing uncertainty
measurements in the
predictions is extremely
powerful capability of
Beijing nuts, a patient neural
networks and a online
incremental learning.
These, you know, just release it.
There's a lot of really good papers there.
It's exciting.
Okay autonomous vehicles.
Oh boy let me try to use as
few sentences as possible
to describe this section of a few slides.
It is one of the most
exciting areas of applications
of AI and learning in
the real world today.
And I think it's the way
that artificial intelligence,
it is the place where
artificial intelligence
systems touch human beings
that don't know anything
about artificial intelligence.
The most hundreds of thousands,
soon millions of cars will be interacting
with human beings, robots, really?
So this is a really exciting area
and really difficult problem.
And there's two approaches.
One is level two where
the human is fundamentally
responsible for the
supervision of the AI system
and level four, or at least the dream
is where the AI system is
responsible for the actions
and the human does not
need to be a supervisor.
Okay, two companies represent
each of these approaches
that are sort of leading
the way Waymo and October,
2018 10 million miles on road today.
This year, they've done 20 million miles
in simulation, 10 billion miles.
And a lot, I've gotten a chance
to visit them out in Arizona.
They're doing a lot of
really exciting work
and they're obsessed with testing.
So the kind of testing
they're doing is incredible.
20,000 classes of structured
tests of putting the system
through all kinds of tests that
engineers can think through
and that appear in the real world.
And they have initiated testing
on-road with real consumers
without a safety driver.
So if you don't know what
that is, that means the car
is truly responsible.
There's no human catch.
The exciting thing is
that there is 700,000,
800,000 Tesla autopilot systems.
That means there's these systems
that are human supervised.
They're using fun, a
multi-headed neural network,
multitask neural network
to perceive, predict
and act in this world.
So that's a really exciting,
real world deployment.
Large scale of neural
networks as a fundamentally
deep learning system, unlike Waymo,
which is a deep learning,
is the icing on the cake
for Tesla deep learning is the cake.
Okay, it's a, at the
core of the perception
and the actions the system performs.
They have to date done over
2 billion miles estimated
and that continues to quickly grow.
I'll briefly mention which
I think is a super exciting
idea in all applications
and machine learning
and the real world, which is online,
so iterative learning, active
learning Andrea Carpathia
who was the head of autopilot,
causes this, the data engine.
It's this iterative process
of having a neural network,
performing a task,
discovering the edge cases,
searching for other edge
cases that are similar
and then retraining the network,
annotating the edge cases,
and then retraining that and
continuously doing this loop.
This is what every single
company that's using machine
learning seriously is doing
very little publications
on this space and active learning.
But this is the fundamental problem.
Machine learning is not
to create a brilliant
neural network, is to
create a dumb neural network
that continuously learns to
improve until it's brilliant.
And that process is especially interesting
when you take it outside
of single task learning.
So most papers are written
on single task learning.
You take whatever
benchmark here in the case
of driving is object
detection, landmark detection,
driving boy area, a
trajectory generation, right?
The, all those have benchmarks
and you can have some
separate and you'll notice
for them that's a single task.
But combining to use a
single neural network
that performs all those tests together,
that's the fascinating
challenge where you're reusing
parts of the neural
network to learn things
that are coupled.
And then to learn things that
are completely independent
and doing the continuous
active learning loop.
They're inside companies.
In the case of Tesla and Waymo in general,
it's exciting to have people,
these are actual human beings
that are responsible for
these particular tasks
that become experts of
particular perception task
expert at a particular
planning task and so on.
And so the job of that
expert is both to train
the neural network and to
discover the edge cases
which maximize the
improvement of the network.
That's where the human
expertise comes in a lot.
Okay and there is a lot of debate.
It's an open question
about which kinds of system
would be which kind of
approach would be successful.
A fundamentally learning based approach
as is with the level two
with the Tesla autopilot
system that's learning
all the different tasks
that are vital involved
with driving and as it
gets better and better
and better, less and less
human supervision is required.
The pro of that approach
is the camera based systems
have the highest resolution.
So the, it's very amenable to learning,
but the con is that it
requires a lot of data,
a huge amount of data and
nobody knows how much data yet.
The other con is human
psychology is the driver behavior
that the human must
continue, continue to mean
remain vigilant on the
level four approach.
That leverage is besides
cameras and radar and so on.
Also leverage is LIDAR on map the pros
that it's much consistent, a
reliable, explainable system.
So the detection, the
accuracy, the detection,
the depth estimation, the
detection of the different objects
is much higher accurate with less data.
The cons is it's expensive.
At least for now, it's less
amenable to learning methods
because much fewer data,
low resolution data
and must require at least
for now some fallback,
whether that's the safety
driver or teleoperation.
The open questions for the
deep learning level to Tesla
autopilot approach is how hard is driving.
This is actually the open
question for most disciplines
in artificial intelligence.
How difficult is driving, how
many edge cases does driving
have can that, can we learn
to journalize over those edge
cases without solving the
common sense reasoning problem?
It's kind of, it's kind of
the task without solving
the human level artificial
intelligence problem
and that means perception.
How hard is perception
detection, intention modeling
a human mental model, a modeling,
the trajectory prediction.
Then the action side, the
game theoretic action side
of balancing, like I
mentioned, fun and enjoyability
with the safety of the systems
because these are life critical systems
and human supervision, the vigilant side.
How good can autopilot get
before vision has detriments
significantly and people fall
asleep, become distracted,
start watching movies, so on and so on.
The things that people naturally do.
The open question is how
good could all autopilot
get before that becomes a serious problem
and if that detriment
nullifies a safety benefit
of the use of autopilot,
which is autopilot AI system,
when the sensors are working
well is perfectly vigilant.
They have, AI is always paying attention.
The open question is for the LIDAR based.
The level for the Waymo
approach is when we have maps,
LIDAR and geo-fenced
routes that are taken.
How difficult is driving the traditional
approach to robotics?
From the DARPA challenge to today
for most of the Thomas vehicle companies
is to just to do HD mass, to use low LIDAR
for really accurate
localization together with GPS.
And then the perception
problem becomes the icing
on the cake because you already
have a really good sense
of where you are with the
obstacles and the scene
and the perception is not
a safety critical task,
but a task of understanding,
interpreting the environment
further so you have more yeah, it's okay.
It's naturally by nature already safer.
But how difficult is it
nevertheless is that problem.
If the perception is the hard problem,
then the LIDAR based approaches is nice.
If action is the hard
problem, then both Tesla
and Wayne will have to
solve the actual problem
without the sensors don't
matter there it's the difficult
problem the planning, the
game theoretic, the human,
the modeling of mental
models and the intentions
of other human beings, the pedestrians
and the cyclists is the hard problem.
And then the other side, the
10 billion miles of simulation,
the open problem from
reinforcement learning,
deep learning in general is how much
can we learn from simulation?
How much of that knowledge can we transfer
to then read the real world systems?
My hope in the autonomous vehicle space,
AI assisted driving space
is to see more applied
deep learning innovation.
Like I mentioned, these
are really exciting areas,
at least to me, of active
learning, multitask,
learning and lifelong learning.
Online learning, iterative learning.
There's a million terms
for it, but basically
continually learning and
then the multitask learning
to solve multiple problems
over the air updates.
I would love to see in
terms of the autonomous
vehicle space, this is common
for, this is a prerequisite
for online learning.
If you want a system that
continues to improve some data,
you want to be able to deploy
new versions of that system.
A test is one of the only vehicles
that I'm aware of in the level
two space that's deploying
software updates regularly
and built an infrastructure
to deploy those updates.
So updating your own networks.
That to me seems like a prerequisite
for solving the problem of autonomy
in the level two space.
Any space is deploy updates
and for research purposes,
public datasets continue.
There's already a few public
data sets of edge cases,
but I'd love to continue seeing
that from automotive companies
and autonomous vehicle
companies and simulators,
Carla Nvidia draft,
constellation voice, deep drive.
There's a bunch of simulators coming out
that are allowing people to experiment
with perception, with planning,
with reinforcement learning algorithms.
I'd love to see more
of that and less hype.
Of course, less hype, one
of the most over-hyped
spaces besides sort of AI
generally is autonomous vehicles.
And I'd love to see real balanced nuanced
in depth reporting by
journalists and companies
on successes and challenges
of autonomous driving.
If we skip any section,
it would be politics,
but me maybe briefly mentioned somebody
said Andrew Yang yang.
So it's exciting for me to see exciting
and funny and awkward to
see artificial intelligence
discussed in politics.
So one of the presidential
candidates discussing
artificial intelligence awkwardly
so that there's interesting ideas,
but there's still a lack of understanding
of fundamentals, artificial intelligence,
there's a lot of important
issues, but he's bringing
artificial intelligence
to the public discourse.
That's nice to see, but
it is the early days.
And so as a community that informs
me that we need to communicate better
about the limitation
capabilities of artificial
intelligence and automation broadly.
The American initiative
AI initiative was launched
this year, which is our
governor's best attempt
to provide ideas and
regulations about what does
the future of artificial intelligence
look like in our country.
Again, awkward but important
to have these early
developments, early ideas
from the federal government
about what what are the dangers
and what are the hopes, the funding
and the education required
to build a successful
infrastructure for
artificial intelligence.
This is the fun part.
There's a lot of tech companies
being brought before government.
It's really interesting in terms of power.
Some of the most powerful
people in our world today
are the leaders of tech companies.
And the fundamentals of
what the tech companies
work on is artificial
intelligence systems.
Really recommendation
systems advertisement,
discovery from Twitter
to Facebook to YouTube
is the recommendation
systems and all of them
are now fundamentally based
on deep learning algorithms.
So you have these incredibly
rich, powerful companies.
They're using deep learning
coming before government
that's trying to see awkwardly trying
to see how can we regulate.
And it's, I think the role
of the ag community broadly
to inform the public and inform government
of how we talk about how
we think about these ideas.
And also I believe it's the role
of companies to publish more.
There's been very little
published on the details
of recommendation systems behind Twitter,
Facebook, YouTube, Google.
So all those systems is
very little as published.
Perhaps it's understandable
why, but nevertheless,
as we consider the ethical implications
of these algorithms, there
needs to be more publications.
So here's just a harmless
example from deep mind
talking about the
recommendation system behind
the play store app discovery.
So there there's a bunch of
discussion about the kind
of a neural net that's
being used to propose
the candidate generation.
So this is after you install
a few apps, the generation
of the candidate,
it's shows you ranked the
next app that you're likely
to enjoy installing.
And so there they tried
LSDM and transformers
and then narrowed it down
to a more efficient model
that's being able to run fast.
That's an attention model.
And then there's some,
again, harmless de biasing
harmless in terms of topics.
The, the model learns to bias in favor
of the apps that are shown
and that thus installed
more often as opposed
to the ones you want.
So there are some waiting to
adjust for the biasing towards
the apps that are popular
to allow the possibility
of you installing apps
that are less popular.
So that kind of process and publishing in
and discussing in public I
think is really important.
And I would love to see more of that.
So my hope in this, in the politics space
in the public discourse space for 2020
is less fear of AI and more a discourse
between government and
experts on topics of privacy,
cyber security and so on.
And then transparency
and recommender systems.
I think the most exciting,
the most powerful
artificial intelligence system space
for the next a couple of decades
is recommendation systems.
Very little talked about it seems like,
but they're going to
have the biggest impact
on our society because they
affect how the information
we see, how we learn, what
we think, how we communicate.
These algorithms are controlling us.
And we have to really
think deeply is engineers
of how to speak up and
think about their societal
implications, not just in terms of bias
and so on, which are sort
of ethical considerations
that are really important but stuff
that's like the elephant in the room
that's hidden, which is how controlling
how we think, how we see the world,
the moral system under which we operate.
Quickly dimension and
wrapping up with a few minutes
of questions if there are
any, is the deep learning
courses this year before
the last few years
has been a lot of incredible
courses on deep learning
and reinforcement learning.
What I would very much recommend
for people is the fast AI of
course from Jeremy Howard,
which uses their wrapper around PI torch.
It's to me the best
introduction to deep learning
for people who are here
or might be listening
elsewhere are thinking
about learning more about deep learning.
That's is to me, the best course,
also a paid, but Andrew Ang, everybody
loves Andrew Ang is the
deep learning AI Coursera
course on deep learning is, is excellent
for especially for complete
begin for sort of beginners.
And then Stanford has
two excellent courses
on visual recognition.
So convolution neural
nets originally taught
by Andrew Carpathy and natural language
processing excellent courses.
And of course here at MIT there's a bunch
of courses especially on the fundamentals
on the mathematics linear algebra
and statistics and I have
a few lectures up online
that you should never watch.
Then on the reinforcement learning side,
David silver is one of the greatest people
in understanding reinforcement
learning from deep mind.
He has a great course, an
introduction to reinforcement
learning, spinning up,
and deeper enforcement
learning from OpenAI.
I highly recommend here
just for the slides
that I'll share online, there's
been a lot of tutorials.
One of my favorite lists of tutorials,
which is I believe the best
way to learn machine learning,
deep learning, natural language processing
in general is it's just code.
Just build it yourself, build the models.
Oftentimes from scratch.
Here's the list of tutorials.
Would that link or would 200 tutorials
on topics from deep RL to optimization
to back prop a LSTM accomplishing
or recurrent neural networks?
Everything over 200 of
the best machine learning
NLP and Python tutorials by Robbie Allen.
You can Google that or
you can click the link.
I love it.
Highly recommend the three books.
I recommend of course,
the deep learning book
by a Joshua Benjamin and Ian
Goodfellow and Aaron Kerrville.
That's more sort of the
fundamental thinking
about from philosophy to
the specific techniques
of the deep learning and
the practical grokking
deep learning, which Andrew
Trask will be here Wednesday.
His book, grok and deep learning I think
is the best for beginners
book on deep learning.
I love it.
He implements everything from scratch.
It's extremely accessible.
2019 I think it was published,
maybe 18 but I love it.
And then Francoise Chevrolet,
the best book on a Kerrison TensorFlow
and really deep learning as well
as deep learning with Python.
Although you shouldn't buy it, I think
because he is supposed to
come up with version two,
which I think will cover TensorFlow 2.0
it'll be an excellent book.
And when he's here Monday,
you should torture him
and tell him to finish writing.
He was supposed to finish writing in 2019.
Okay, my general hopes
as I mentioned for 2020,
is I love to see common sense reasoning
and to not necessarily enter the world
of deep learning, but be a
part of artificial intelligence
and the problems that people
tackle as I've been harboring
active learning is to
me is the most important
aspect of real world
application of deep learning.
There's not enough research.
There should be way more research.
I'd love to see active
learning, lifelong learning.
That's what we all do as human beings.
That's what AI systems need to do.
Continually learn from
their mistakes over time,
start out dumb, become
brilliant over time.
Open domain conversation,
with the Alexa prize.
I would love to see breakthroughs there.
Alexa, folks thinks we're still two
or three decades away,
but that's what everybody
says before the breakthrough.
So I'm excited to see
if there's any brilliant
grad students that come
up with something there.
Applications in autonomous vehicles
and medical space, algorithmic ethics.
Of course, ethics has
been a lot of excellent
work in fairness, privacy and so on.
Robotics and as I said,
recommendation systems.
The most important in terms of impact part
of artificial intelligence systems.
I mentioned soup in terms of progress,
there's been a little bit of tension,
a little bit of love online
in terms of deep learning.
So I just wanted to say
that that kind of criticism
and skepticism about the limitations
of deep learning are really
healthy in moderation.
Jeff Hinton, one of the
three people to receive
the touring award, as,
as many people know,
has said that the future
depends on some graduate
student who is deeply suspicious
of everything I have said.
So that's suspicion.
Skepticism is essential, but in moderation
just a little bit.
The more important thing is perseverance,
which is what cheffy Hinton and the others
have had through the winters of believing
in your own nets and an
open mindness for returning
to the world of symbolic AI.
Oh, the expert systems of complexity
and cellular automata of
old ideas in AI and bringing
them back and see if there's ideas there.
And of course you have to
have a little bit of crazy.
Nobody ever achieves something brilliant
without being a little bit of crazy.
And the most important
thing is a lot of hard work.
It's not the cool thing these days,
but hard work is everything.
I like what JFK said.
How about us going to the moon?
Us, I was born in the Soviet union.
See how I conveniently just
said us going to the moon
is a, we do these things
not because they're easy,
but because they're hard.
And I think that artificial intelligence
is one of the hardest and
most exciting problems
that are before us.
So would that like to thank you
and see if there's any questions?
(applauding)
- [Student] In the 1980s
parallel distributed
processing books came out.
They had most of the
stuff in it back then.
What's your take on the roadblocks?
The most important vote blocks apart
from maybe funding?
- I think fundamentally,
I mean they're well known
as limitations is that
they're really inefficient
at learning and there are
not, so they're really good
at extracting representations
from raw data,
but not good at learning knowledge bases
of like accumulating knowledge over time.
That that's the fundamental
limitation I ask for.
Systems are really good
at accumulating knowledge,
but very bad at doing
that in an automated way.
Symbolic AI, so I don't
know how to overcome
a lot of people say
there's hybrid approaches.
I believe more data, bigger networks
and better selection of data
will take us a lot farther.
- [Student 2] Hello, Alex.
I'm wondering if you recall,
what was the initial spark or inspiration
that drove you towards work in AI?
Was it when you were pretty young
or was it in more recent years?
- So I wanted to become a psychiatrist.
I wanted to I thought of
it as kind of engineering
the human mind by sort of manipulating it.
I thought that's what
I thought of psychiatry
is by using words to sort
of explore the depths
of the mind and be able to adjust it.
But then I realized that psychiatry
can't actually do that.
And modern psychiatry is more
about sort of bioengineering, this drugs.
And so sort of, I thought
that the way to really explore
the engineering of the
mind is the other side
is to build a sort of and that's also when
a C plus plus really
became the cool hot thing.
So I learned to program at 12
and then never look back
hundreds of thousands
of lines later.
Just I love program, I love building.
And that's to me is the
best way to understand
the mind is to build it.
- [Student 3] Speaking of
Belgian mind, do you personally
think that machines will
ever be able to think,
and the second question,
will they ever be able to feel emotions?
- 100% yes.
100% they'll be able to think
and they'll be able to
feel emotions because,
so those concepts of thought
and feeling are human
concepts and to me, okay,
they'll be able to fake it.
They're there for,
there'll be able to do it.
Like I've made a, I've
been playing with Roombas
a lot recently, Roomba vacuum cleaners.
And so I've now started having Roombas
scream like, like there's like moaning
in pain and they became,
I feel like they're having emotions.
So like the faking creates the emotion.
Yeah, so the display of
emotion is emotion to me.
And then display a thought is thought.
I guess that's the sort of everything
else is impossible to pin down.
- [Student 3] I'm asking.
So what about the ethical aspects of it?
I'm asking it because I will burn
into Soviet union as well.
And one of my favorite recent
books is Victor Parliament's
IFAC and it's about AI feeling emotions
and suffering from it.
So I don't know if you've read that book.
What do you think about
AI feeling emotions
in that context or in
general ethical aspects?
- Yeah, it's a really difficult question.
Answer is yes.
I believe AI will suffer
and it's unethical to AI,
but I believe suffering exists
in the eye of the observer.
Sort of like if a tree
falls and nobody's around
to see it, it never suffered.
It's us humans that see
the suffering in the tree
and the animal and our
fellow humans and sort of
in that sense the first time a programmer
with a straight face delivers a product
that says it's suffering is
the first time he becomes
unethical to torture AI systems.
And I can do, we can do that today.
Like I already built the Roombas I,
they won't sell currently,
but I think the first time
a Roomba says, please don't hurt me.
That's when we start to
have serious conversations
about the ethics and it's,
it sounds ridiculous.
I'm glad this is being recorded
because it won't be ridiculous.
And just a few years.
Yeah.
- [Student 4] Is a reinforcement learning
a good candidate for achieving
a general artificial intelligence?
Are other and are any other,
are there any other
good candidates around?
- So yeah, to me the answer
is no, but it can teach
us some valuable gaps that can
be filled by other methods.
So I believe that simulation is different
than the real world.
So if you could simulate the real world,
then deep RL, any kind
of reinforcement learning
with deep representations would be able
to achieve something incredible.
But to me the simulation is very different
than the real wall.
So you have to interact in the real world
and there you have to
be much more efficient
with learning and to be more
efficient with learning,
you have to have ability
to automatically construct
common sense, like common
sense reasoning seems
to include like a huge
amount of information
that's accumulated over time.
And that feels more like
programs than functions.
I like how like a skewer
talks about deep learning
learns functions
approximators deep RL learns
an approximator for policy,
whatever, but not programs.
It's not learning a thing that's able
to sort of that's essentially
what reasoning is a program.
It's not a function.
So I think, I think no,
but he'll continue to,
one inspires and to inform us about where
the true gaps are.
I think the ability to,
but I'm so human centric,
but I think the approach of being
able to take knowledge and put it together
sort of building into
more and more complicated
pieces of information concepts,
being able to reason in that way.
There's, there's a lot of methodologies
that all schools sort of that's the falls
under the ideas of symbolic AI of doing
that kind of logic, reasoning,
accumulating knowledge basis.
That's going to be an essential part
of general intelligence.
But also the essential part
of general intelligence
is the Roomba that says I'm intelligent
F-you if you don't believe
me, like a very confident,
like, cause right now,
like Alexa is very nervous.
Like, Oh, what can I do for you?
But once Alexa says,
like, you know, is upset
that you would like turn her off or treat
her like a servant or say that
she's not intelligent, that's that.
That's where the
intelligence starts emerging.
Cause I think he was a pretty dumb
and what general we're
all like intelligence
is in a, it's a very kind
of relative human construct
that we've kind of
convinced each other's of.
And the, and once AI
systems are also playing
that game of creating constructs
and that human communication
that's going to be important.
But of course for that you
still need to have pretty
good witty conversation.
And for that you need to do
this symbolically I think.
- [Student 5] I'm wondering
about the autonomous vehicles,
whether they are responsive
to environmental sounds.
I mean if I notice in her
car autonomous vehicle
driving erratically
will respond to my beep.
- That's really interesting question.
As far as I know no I think Waymo hinted
that they look at sound a little bit.
I think they should.
So there's a lot of stuff
that comes from audio
that's really interesting.
The sort of Waymo have said
that they use audio for sirens.
So detecting sirens from far away.
Yeah, I think audio is a lot
of interesting information.
Like the sound that the car,
the tires make on different kinds
of roads is very interesting.
We kinda, we use that
information ourselves too,
depending on kind of like off-road.
A wet road when it's not
raining sounds different
than dry road.
So there's a lot of
little subtle information,
pedestrians, yelling
and that kind of stuff.
It's actually very
difficult to know how much
you get from audio.
Most robotics folks think
that audio is useless.
I'm a little skeptical.
Yeah but nobody's been able to identify
why audio might be useful.
- [Student 6] So I have two questions.
My first is what do you
think is the ultimate sort
of end point for super
machine intelligence?
Like we'll be sort of be
relegated to some obscure
part of the earth, like
we've done some next primates
and next intelligent primates.
And my second question
is, should there be,
should we have equal rights
for beings made out of Silicon
versus carbon for
example, like, like robots
- Separates or same?
- Equal rights with humans?
- Yeah so the future
of super intelligence,
I think I have much less work.
I see very much fewer paths
to AI, AGI systems killing humans
than I do for systems living among us.
So I think I see exciting
exciting or not so exciting
but not harmful futures.
I think it's very difficult
to create AI systems
that will kill people
that aren't like literally weapons of war.
They're like, it'll always
be people killing people.
Like the things we should be worried
about is other people.
That's the fundamental.
So there's a lot of ways nuclear weapons,
there's a lot of existential
threats to our society
that are fundamentally human at the core.
And AI will be, might be tools of that,
but there'll be also tools
to defend against that.
I also see AI proliferating as companions.
I think companionship will
be a really interesting,
like we will more and more live
as we're already doing the digital world.
Like you have an identity
on Twitter and Instagram,
especially if it's anonymous or something.
You have, you have this
identity that you've created
and that will continue growing more
and more, especially for people born now
that it's kind of this artificial identity
that we live much more
in the digital space
and in that digital space
as opposed to physical space
is where AI can thrive
much more currently.
It'll thrive there first.
And so we'll live in a world
with a lot of intelligent
first assistants,
but also just intelligent agents.
And I do believe they should have rights.
And in this contentious
time of people groups
fighting for rights, I
feel really bad saying
they should have equal rights.
But I believe that I've I've talked to,
if you read the work of
Peter singer of looking,
I like, my favorite food is steak.
I love meat, but I also feel horrible
about the torture of animals.
And that's, that's the
same kind of, to me,
the way our society thinks
about animals is a very similar way.
We should be thinking about robots
or we will be thinking about robots.
And I would say about 20 years.
- [Student 7] So one, one final question.
Yeah well they become our masters.
- No, they will not be our masters.
What I'm really worried about is, well,
who will become our masters are owners
of large tech companies
who use these tools
to control human beings
first, unintentionally
and then intentionally.
So we need to make sure
that we democratize AI.
It's the same kind of same kind of thing
that we did with government.
We make sure that we at the
heads of tech companies,
if maybe people in this room will be heads
of tech companies one day.
We have people like George Washington
who relinquished power at
the founding of this country.
Forget, I forget all the
other horrible things
he did, but he relinquished
power as opposed
to Stalin and all the
other horrible human beings
who have sought instead absolute power,
which will be the 21st century.
AI will be as the tools
of power in the hands
of 25 year old nerds who
should be very careful
about that future.
So the humans will become
our masters, not the AI.
AI will save us.
So on that note, thank you so much.
(applauding)
