The R&D activities
carried out each day
by this audience pave the way
to the hot topics of tomorrow
from the Internet
of Things to 400G
to optic traffic
integrations, our work
sets the stage for
future innovations
and for a better user experience
for customers everywhere.
In fact, it was
just a few years ago
when we were working
on ways to achieve 5G.
And now, we have papers
on field trials and demos
that mean we're
ready to roll it out.
And that deployment
should play a key role
in supporting the needs of our
next visionary speaker, Dr.
Dmitri Dolgov.
Dmitri is currently Chief
Technology Officer at Waymo.
He was one of the
original members
of the Google self-driving
car project, which
became Waymo in 2016.
Prior to Waymo, he worked on
self-driving cars at Toyota
and at Stanford as part of
the Stanford's DARPA Urban
Challenge team.
In 2008, the IEEE
Intelligent Systems magazine
named Dmitri one of the AI's
10 to Watch, the future of AI.
Waymo is currently
the only company
with a fleet of fully
self-driving cars
on public roads.
A suite of sensors including
lidar, radar, and cameras,
gives Waymo's vehicles
a 300 degree view
and a detailed 3D
picture of the world.
Today, Dmitri will be
discussing Waymo's mission
to make it safe and easy
for people and things
to move around.
Please join me in
welcoming Dmitri Dolgov.
[APPLAUSE]
Good morning, everyone.
Thanks for having me here.
Now, the idea of a self-driving
car is hardly a new one.
In fact, it might be as old
as the motor car itself.
However, the story of these
cars becoming a reality
is a fairly new one.
So today, I'm here to talk about
the future of the technology.
But I'd like to start by
revisiting the recent past.
Because it was almost exactly
10 years ago in January 2009
when a small group
of us, about a dozen,
came together and started
the Google self-driving car
project.
Now, at that time, a
small group of students,
academics, industry
experts, had just
finished a race of autonomous
vehicles called the DARPA Urban
Challenge.
And I was fortunate to
be involved at the time.
Now, I've got to say, those
early days were pretty amazing.
You get to learn
about a new problem.
And it makes tremendous progress
in a very short amount of time.
However, while this
DARPA-sponsored competition
was a tremendous success, it
was still incredibly early days
in the industry.
The technology was
in its nascent form.
And there was no
self-driving car industry
to speak of that could
push this forward.
So fortunately at
that time, Google
recognized the transformative
nature of this technology
and started the project.
So when we began, our goal was
to take this early research
work and take it from the
research prototype stage
to the next level.
And what drove us then, and
what still motivates us today,
is the fundamental belief
that self-driving cars
can make our roads safer.
Today, worldwide, more
than 1.3 million people
die on the roads every year.
This is an insane number.
That's about 150
people whose lives
are lost to traffic accidents
every hour of every day.
This is more than
two every minute.
We can do a lot
better than that.
Now, we also know
that about 94 percent
of the accidents on the roads
today are due to human error.
People get drunk.
People get distracted.
People text.
Self-driving cars don't.
Now, what's more,
this technology
has the potential to
make transportation
more accessible for everyone.
So when we started this project,
in the very first phase of it
in 2009, our first
order of business
was to learn as much as we
can about the problem space.
So to that end, we
created a couple
of milestones for ourselves.
One was to drive 100,000 miles
in total in autonomous mode.
And at that time, that
was orders of magnitude
more than anybody
has done at a time.
The second milestone was
to drive these 10 100 mile
routes in full autonomy
from the very beginning
to the very end with
zero human intervention.
So here are some
videos of those routes.
So these routes
were very carefully
selected to be extra challenging
and covered the full complexity
of the driving task.
You can see we drove at night.
We drove during the day.
We drove on freeways.
We dealt with some
construction zones.
We dealt with some
urban environments.
And then, yeah, this is, for
people who are not from the US,
this is an famous
or infamous Lombard
Street in San Francisco.
So that time in 2009, we
were able to take advantage
of the modern
developments in sensing
technology compute and most
important software algorithms
to accomplish these milestones
in just under two years.
And in the process, we
developed a better understanding
of the problem and the full
complexity of the task.
But at the same time,
we gained confidence
to pursue it further marking
the second phase of the project.
Now, in the second
phase, our goal
was to bring this
technology to life.
And in 2015, we were
able to complete
the world's first
fully self-driving test
ride in Austin, Texas.
And we did it in a
custom-designed vehicle
that you see here.
It was specifically
engineered for this task.
In fact, it had no
steering wheel or pedals.
We called it the Firefly.
And the man that you see in
this picture is Steve Mann.
And what made a trip
particularly remarkable
is that Steve
happens to be blind.
Let's be clear.
During that ride
that he took in 2015,
it was a completely uncontrolled
environment on public roads.
There was no chase
vehicle, no police escort,
no test driver on
board to help out.
It was just Steve in a car with
no steering wheel or pedals,
by himself, taking a ride
from a doctor's office
to a local park.
Now, as Waymo, we're
on to our next phase.
So building on that
second trip of 2015,
we've expanded to operate a
fleet of self-driving cars
in Phoenix, Arizona.
And we've been test driving them
since 2017 in driverless mode.
And today, you have people
who are using these cars
to get around every day.
They take them to work.
They take them to school.
They use them to run errands.
And we've even begun giving
some members of the public
a chance to experience their
first truly driverless rides
with no one in
the driver's seat.
So for these people, what
seemed like science fiction
a while ago is
their reality today.
Let's take a look.
OK, day one of self-driving.
Are you ready?
Yeah.
Go.
Oh, this is weird.
[LAUGHTER]
Yeah.
This is the future.
Yeah, she was like, is there
no one driving that car?
[LAUGHTER]
I knew it.
I was waiting for it.
[MUSIC PLAYING]
You'd certainly never
know that there wasn't
someone driving this car.
Yo, car.
Selfie.
Thank you, car.
Yeah, thank you, car.
The one thing I
want to emphasize
is that unlike other technology
that you might have in your car
today where there's still a
driver that is required to take
control every once
in a while, Waymo
is always the driver
from beginning to end.
And as you can see in this
video that we just launched,
no one is required to
sit in the driver's seat.
In fact, there doesn't need to
be a person behind the wheel
or in the car at all.
Now, the Society of
Automotive Engineers
have developed a scale to
distinguish between that.
They call them the levels
of autonomy from L1 to L5.
So levels L1 through
L3, there still
needs to be a
driver in the loop.
So these systems are
really driver assist ones.
But levels 4 and 5 is
where it gets interesting.
There is a step function,
step transition,
where you really need to employ
the full power of modern AI
to completely
replace the driver.
That means an empty car
can come pick you up
and letting you travel
door to door all
the way to your destination
while you're just
sitting in the passenger
seat enjoying the ride.
This is truly driverless.
And at Waymo, we like to
say that our goal is not
to build a car.
We're building a driver.
Our mission is to make it safe
and easy for people and things
to move around.
And we're doing that by building
the world's most experienced
driver.
And while we think that
ride hailing service
is a great way to introduce
more people to this technology,
we also believe
that this technology
can have transformative
impacts across transportation
more broadly.
You can have a Waymo
driver transport goods.
You can have them connect
people to public transit.
Or it can shuttle you around
in your own personal vehicle.
Now, this is a
difficult problem.
But we're able to do
this, to a large extent,
because at Waymo, we have
taken an integrated approach
to hardware and
software development.
So rather than, on the hardware
side, relying on off the shelf
modules that might have been
designed for something else,
our suite of sensors
and computers
is designed from the ground
up specifically for the task
of autonomous driving.
So let's take our
lidars for example.
We've designed three
types of different lidars
for the task of full autonomy.
On top of our vehicles-- this
is the picture you see here--
we have our medium
and long range lidar
that give us 360 degree
coverage around the vehicle
in all directions.
Our long range
laser is so powerful
that it can see up
to 300 meters away
in high resolution, roughly
the length of three football
fields.
Down below around the
perimeter of the car,
we have four short range lasers.
And their intent is to
cover the blind spots
so that we can detect
small objects right
next to our vehicle.
Now, lidar works by
painting a precise 3D model
of the world around us
by sending out millions
of laser pulses every second.
However typically, lidars
have been used mostly just
to measure distance to objects.
So they measure time of flight.
And you get distance to whatever
that pulse happened to hit.
But because we design
our own lidars,
we're able to extract
much richer, raw data that
can come in very handy
in a variety of scenarios
and particularly when we're
dealing with bad weather.
Our vision system is made
up of 19 cameras that
provide 360 degree coverage
of high resolution color
imagery around the vehicle.
And because our vehicles operate
at all times of day and night,
our cameras have
high dynamic range
that allows them to see a wide
variety of lighting conditions,
anything from an unlit parking
lot in the middle of the night
all the way to driving into
a blazing sun at sunset.
So you may be wondering
what's going on in this video.
And yeah, while our cameras
are pretty high tech,
we do use a fairly
low tech solution
to make sure that we're
equipped to deal with birds.
And this is what's being tested
here, cleaning simulated bird
poop fairly successfully.
Lots of little
things like that you
have to take care of
when you are putting
a product out there,
not just doing
research and early development.
Now, also on our cars, we
have an array of radars.
And unlike common
automotive radars
that you have on many
vehicles in the market today,
our system has a 360
degree integrated field
of view, which allows
us to seamlessly track
objects all around our vehicle.
And the radars also nicely
complement the other sensors
we have in the car
by being highly
effective in different
weather conditions
like rain, fog, or snow.
Now, we've designed
these powerful sensors
to give us lots
of rich, raw data
so we can use powerful modern
AI to do late sensor fusion.
Now, this means we need a
lot of computational power
to chew through all
of that in real time.
And that's why we designed
our own compute platform.
So our engineers
have put together
a computer that's specifically
tailored to our needs
and utilizes everything
from general purpose CPUs
all the way to custom FPGAs
and everything in between.
So we get the biggest bang for
the buck for our application
in terms of watts and dollars.
And internally,
our engineers like
to joke that the computer
that we have on the car today
compares well to
some of the ones
that we've seen on a
top 500 supercomputer
list in recent history.
And to give you an idea
of just how fast compute
has been growing
onboard the vehicle,
just in a recent few years,
the amount of operations
per second that
are available to us
have grown by about
a factor of 50.
So that's our main
hardware, but we also
have some other
supplementary sensors
in the vehicle
such as microphones
that allow us to
detect sirens and cell
modems that allow our cars to
communicate with one another.
And I'll talk a little bit
about how they share information
later.
But driving is a real-time
and highly dynamic activity.
And because we need to make
decisions in real-time,
and it's a safety
critical environment,
all of our vehicle
driving decisions
are performed onboard.
So while we don't
rely on cell networks
to make driving decisions,
we can leverage it
for expanded capability.
So the promise of high bandwidth
and low latency networks
is something that we're
very interested in
and it's very welcome.
So we're definitely
keeping an eye
on the great work
that's happening
in this field, some of
it coming from the people
in this very room.
Now, when it comes
to software, AI
touches every part
of our system.
And to be a capable
and safe driver,
our cars need to have a
deep semantic understanding
of the world around them.
And today, I want to cover three
main areas where AI has played
a major role in getting our
cars the capability to be truly
self-driving--
perception, prediction,
and planning.
So first, let's talk
about perception.
You need to detect and
classify objects around us.
The task of seeing
the world around you
is pretty fundamental
to driving around.
Second prediction-- we need to
interpret the movements of all
of those objects, reason
about their intent,
and predict what they
will do in the future.
We also need to understand
how all of these objects
might interact with one another.
And finally, planning--
our cars need
to use all of that information
to act in a smooth, safe,
and predictable manner.
So let's start with perception.
Now, detecting and
classifying objects
is a key part of driving.
But there are many reasons
why this can be challenging.
So today, I want to highlight
three main axes of complexity.
First, there can be
a lot of variability
in object appearance.
A classic example is people.
Now, pedestrian
come-- pedestrians
come in all kinds of
shapes, postures, and sizes.
So here's some examples.
On the left, you see some people
wearing dinosaur costumes.
In the middle, you see somebody
carrying a large plank of wood.
And on the right side, there
is a construction worker
that is half occluded
by a hole in the ground.
So examples in the
middle and on the right
are particularly
interesting because there's
a lot of occlusion going on.
So there's less data for
us to make an accurate
classification.
But luckily, our high
resolution sensors,
and lots of data that we can
use to train our ML models,
helps us correctly
detect pedestrians
in all of these cases
The second axis of complexity
that I want to talk about
has to do with the environment.
Now, environmental
factors affect
perception because
of interference
with sensors and
reduced visibility.
So rain and snow
pose challenges.
But so do many other factors
such as dust or sun glare.
Now, the third
axis of complexity
that I want to talk to
you about today when it
comes to perception is context.
It's not enough to simply
detect and classify an object.
Our vehicles need to
understand context.
This is something that we call
scene level understanding.
So let's take a look
at this example.
There are four stop signs.
And sure, we can
detect each one.
But what does each one of
them mean and how should it
affect our behavior?
So example on the top left, this
is your very typical stop sign.
It just controls
an intersection.
We know what to do with that.
The example below-- there is
a stop sign and a school bus.
So you have to detect both.
And you have to understand
the relationships between them
so that you can
behave accordingly.
The example on the top right
is a person, a crossing guard,
often happens in school zones,
that's carrying a stop sign.
So we need to understand the
situation where this stop
sign applies to us when it's
relevant, in other cases, where
we should filter it out.
And this last example
on the bottom right
is an interesting, pretty
rare one, although we actually
see this more often
than you would think,
is a person who is just biking
around holding a stop sign.
And the right answer
there, of course,
is just to ignore the
stop sign completely.
Now, let's talk about
prediction next.
So we can detect and classify
all the relevant objects
around us.
But we also need to
predict what they're
going to do in the future.
Now, at a basic
level, we can do that
by understanding the
rules of the road,
understanding the
different road users
and their unique behaviors, like
pedestrians, cyclists, cars,
and then, of course,
being able to process
real-time information such as
their trajectory and speed.
So this works great when people
actually follow the rules.
But we all know that's
not always the case.
So here's an example
of a red light runner.
Now unfortunately, we
see this kind of thing
more often than we'd like.
Now, let me break this down
from the car's point of view.
Our vehicle is about
to proceed straight
through an intersection.
We have a clear green light.
And the cross traffic is
stopped with a red light.
But just as we're about to
enter in the intersection, all
the way in the right corner,
we see a vehicle coming fast.
Now, our models
understand that this
is unusual behavior
for a vehicle that
should be decelerating.
So we've predicted the car
would run the red light.
And we preemptively
slowed down, which
you can see here with this
red fence across our path.
So this gives the
red light runner
enough room to
pass in front of us
while it barely avoids
hitting another vehicle.
And we can detect
this kind of anomaly
because we've
trained our ML models
using millions of real
world examples of real life
interactions.
Now, this was an example
of just one car doing
something anomalous while kind
of ignoring everything else
that's going on in the world.
But driving, of course,
is a very social task.
So it's important
for us to understand
how different objects in the
road interact with one another.
Let's take a look at this video.
So here, it's important to
understand that as we approach
the parked car, that
parked car may affect
the behavior of other people.
In this case, there are
two cyclists on the right.
And we need to anticipate
well in advance
that these two cyclists will
want to merge in front of us.
Now, let's talk about planning.
This last video is
interesting not just
because it illustrates the
need for accurate perception
and prediction models,
but also because it
highlights the complexity
of making decisions.
So in this case, there
are multiple things
that our car could have done.
Slowing down was not the
only possible decision.
So let's go back to
that part of the video.
Now, as we approach
the parked car,
there are actually a
number of potential moves
our car could have made.
It could have sped up to pass
in front of the cyclists.
It could have slowed
down to let them in.
Or it could have changed
lanes to avoid the interaction
altogether.
Now, in this situation,
changing lanes
might seem like a good answer.
But there's actually
a fast moving vehicle
that's approaching from behind.
And at that distance
and relative speed,
that driver probably would
not have been too happy
with us cutting them off.
So part of being
a good driver is
being able to provide
a safe and smooth ride
while being predictable to
all of those people around us.
And the right decision
depends on the specifics
of each situation as well
as a deep understanding
of the social
conventions of driving
and an understanding
of the expectations
that other actors in the
scene have of your behavior.
And with more than 10 million
miles of our real world
experience as well as
several billion miles
of simulated driving that we can
then use to train and validate
our AI algorithms,
our vehicles are
capable of handling all kinds
of real and complex situations
whether it's two dogs
going for a midnight run,
or a jogger that jumps into
the road chasing after her dog,
or the hectic weaving of
traffic on San Francisco roads,
or a crossing guard
and a policeman
that are controlling an
intersection in a school zone.
So this is just a
very small sampling
of things we encounter with our
cars operating on public roads.
Now, one benefit of having
a fleet of self-driving cars
is that whatever
one car learns, it
can pass onto the entire fleet.
So this, of course, happens
whenever we see something
interesting in one car.
And then our
engineers work on it.
They make some improvements.
And we push an update
to the entire fleet.
But it can also happen
dynamically with changes
to our behavior in real time.
So let's take a look
at this example.
Here, this is in
Phoenix, Arizona,
you see one of our cars is
traveling in the right lane.
And it's about to come up
on a construction zone.
So you'll see this coming
into view in just a second.
So it's seeing some cones.
It's seeing a
construction truck.
So it does the right thing.
It understands the semantics
of all those objects.
It slows down and changes lanes
to the left, passes around it.
So all of these decisions
are being made in real time
by the software onboard the
vehicle, all the perception,
prediction, and planning.
Now, let's take another look to
see how our car is discovering
this information.
So as these objects come
into the field of view,
it first doesn't completely
understand what's going on.
Then it builds up the model
of the construction zone
and understands the
exact geometry of it
and handles it appropriately.
But once it's built up that
model of the construction zone,
it then shares this information
automatically in real time
with the entire fleet.
So now let's take a look at
what happens when another Waymo
vehicle turns onto the same
road about an hour later.
You see that at this time,
as soon as our vehicle turns
onto the road, it preemptively
changes into the left lane.
That same morning again, another
one of our vehicles, and it's
already in the left lane,
while other human drivers
are forced to slow down and
merge at the last second.
So you can imagine how handy
this kind of capability
would be if there was
an actual road closure.
Our vehicles would be able
to take an alternate route
without having to run
into the same roadblock.
Now today, Waymo vehicles
number in the hundreds.
But as more autonomous
cars join the road,
there's an incredible
potential for entire fleets
of cars to work together
and share sensing prediction
and planning information.
And this ability could
represent another major leap
that will allow us
to even better tackle
the complexities of
real world driving.
For perception, vehicle to
infrastructure communication
could be a wonderful
opportunity.
One example is traffic lights.
So instead of us using
cameras to detect their state,
they could just
share it with us.
Or even better, they just
share their schedules
so we can plan ahead.
For prediction, the task of
signaling and understanding
intent becomes
easier if cars can
talk to each other and share
not just their current state,
but also what they're
intending to do in the future.
And for planning, we can use
communication between cars
to make our transportation
system more efficient.
We can reduce the
gaps between the cars.
We can optimize traffic
globally in these fleets of cars
to reduce traffic congestion.
And while advances in
artificial intelligence today
mean that we're able to
build fully self-driving cars
and have them on
the roads, our goal
is to bring this technology
to everyone wherever they are.
And there are a lot of
exciting developments
in the field of
communications that
could make this technology
even more capable whether we're
talking about optical
communication and lidars,
moving data around
in our computers,
data processing in the
back end of data centers
so we can train even
more advanced machine
learning models, or real-time
communication between cars
in a fleet that's been deployed.
So all of that ultimately
means safer roads.
So I very much look
forward to working more
with this community.
Thank you.
[APPLAUSE]
[MUSIC PLAYING]
