DAVE FERGUSON: The other Google
X moonshot that I'd
like to speak about tonight
is self-driving cars.
So these are vehicles that can
navigate without any human
input whatsoever.
And when I talk about this
project to people, they
sometimes respond,
I can drive.
I like driving.
What's the point?
Why aren't you guys working
on something more useful?
So indeed, what is the point?
Well, for starters, time.
In the US, we spend an average
of 50 minutes a day commuting
to and from work.
In New Zealand, we drive
an average of 30
kilometers every day.
And this is an incredible amount
of time that could
better be spent working,
sleeping, relaxing, watching
cat videos on YouTube.
Self-driving cars could
give this time
back to their occupants.
Now, it's estimated in the
US just this time due to
commuting is a wasted 50
billion hours a year.
Secondly, they could also be
more efficient in how they use
the roads that we
already have.
So they could drive
closer together.
They could also drive
on narrower lanes.
And a recent estimate has
shown that just this
optimization could save 10
billion liters of fuel
a year in the US.
Secondly, they could be a lot
safer than the current
vehicles that we have.
Worldwide, we lose over a
million people every year to
traffic accidents.
It's the number one cause of
death for young people in the
developed world.
And over 90% of these
accidents are
due to human error.
Imagine if you had vehicles
that never made mistakes,
always paid attention, never
tried to tweet while eating a
Big Mac and drive all
at the same time.
Now, these are two of the
perhaps more obvious examples
of where driverless cars
would benefit society.
But this is really only
scratching the surface.
Imagine not needing parking
spots or parking lots at any
places of interest.
Imagine not needing to own a car
because there's always one
available on demand for a
fraction of the cost.
Imagine vehicles that didn't
get in collisions, so they
didn't need heavy collision
reinforcement, which meant
that they could be much, much
lighter and thus, much, much
greener than the cars
we drive today.
And then imagine driverless cars
that could run errands
with or without people
on board.
Every time we have come up
with some new technology,
society has found ways to
make the most of it.
Think of the internet or GPS as
two very recent examples.
And yet, in general, humankind
has a glorious tradition of
not imagining what things
could be like, of being
anchored to how things are
now and thinking only
incrementally.
So we look at the accidents
that we have from driving.
We look at the time that's
wasted commuting.
And we realize that these are
inefficiencies, but we accept
it as the tradeoff that we make
for the convenience of
driving, our current
relationship with the vehicle.
But why should we accept this
when we could do so, so much
better potentially?
And then finally, does
anyone disagree that
they're kind of cool?
I mean, who wouldn't want
KITT from Knight Rider
driving them around?
Now the idea of tackling
driverless cars may seem a
little bit crazy, a little
bit far-fetched.
But maybe it's just far-fetched
enough to be worth doing.
And certainly, not everyone
shares this opinion.
Even some heads of car companies
believe that this is
never going to happen, that it's
an impossible technology.
But to me, that's even
crazier than the idea
of driverless cars.
It's impossible that
this isn't going to
happen at some point.
And of course, there
are hurdles.
There are lots of hurdles.
I find it hard programming
my TV remote.
Imagine how hard it is to get
a car to drive itself.
And beyond the technology issues
are the policy issues,
the liability issues, the
regulatory issues.
But that's OK.
Rarely is anything worth
doing ever that easy.
So we decided to give it a shot
at Google X. And we asked
ourselves how can we try to
solve this problem that has
seemed for a long time
to be intractable?
And the first idea was to put
all of the intelligence and
the sensing on the
vehicle itself.
So a lot of current and previous
work has focused on
changes to the infrastructure,
to the environment.
So cars can be equipped to talk
to each other and tell
each other where they are
and what they're doing.
Cars can talk to
traffic lights.
And the traffic lights can tell
them what the color of
the light is and when
they should go.
Cars can track markers or
magnets in the road to tell
them where they should drive.
Now, this is all fantastic work,
and it simplifies the
problem enormously.
But it means that we have to
wait for this infrastructure
to be available before we can
unleash this technology and
the promise of driverless
cars.
And we've already been
waiting a long time.
There was a very successful
research project where they
put magnets in a road in a
highway in California and had
cars track it.
That happened in
the early '90s.
So we've been waiting
a long time.
And if we can put all of this
intelligence and sensing on
the vehicle itself and not need
to rely on any changes in
the environment, then we don't
need to wait anymore.
The problem is that's
really, really hard.
So some of the sensing that we
might put on would be radars
that can detect where other
vehicles are and what speed
they're going, lasers that can
generate a three-dimensional
representation of everything
around the vehicle, and
cameras to detect lights and
signs, traffic lights, tail
lights, markers on the road.
But perhaps, even though this
seems like a very, very hard
way of solving the problem,
maybe there are other things
that we can do to make
it slightly easier.
And the first idea of making it
easier is through mapping.
So what if we were to map the
entire world and then use that
map to tell us what we should
be doing at every
point in the world?
Now, at first, this also might
seem a little crazy.
And to be honest, this was one
idea that I thought was a
little nutty to begin with.
But what if we were to give it
a go, and then see what that
allowed us to do that we
couldn't do otherwise?
And maybe we can figure out how
to do this mapping itself
later efficiently.
After all, we do have a couple
examples of companies that are
able to keep world-size maps up
to date pretty effectively,
wonderful companies with
wonderful maps.
The second idea that we had was
to focus and simplify the
problem, and then simplify
it some more.
Now, you might think that
in order to drive in an
environment, you need to
understand everything that's
going on in that environment to
make the right decisions.
But this is an incredibly
hard robotics problem.
It basically means that we need
to solve what's known as
the artificial intelligence
problem, where the robot has
to have full common knowledge
and common sense to reason
about anything that
could happen.
But in fact, the act of driving
requires coming up
with exactly two values--
what angle to have your steering
wheel, and how hard
to push the gas or brake.
That's it--
two numbers.
So perhaps we can take all of
this complexity that's in the
environment, and we can filter
it down to just the key
components that really make a
difference in us coming up
with those two small numbers.
And so that's sort of what
we've tried to do.
So we take everything that's
in the environment, all the
vehicles, all the pedestrians,
all the static objects, and we
try to filter it down to only
the things that matter.
And then we only consider
those for our task.
And basically, once we have this
problem transformed to be
as simple as possible, we get
a bunch of smart people
together, and we try to tackle
the bits that remain.
And ideally, people that don't
know any better, that don't
realize just how hard
this problem is.
And that's what we've done.
We have a fleet of vehicles that
have driven over 700,000
kilometers, mostly in the
northern Bay Area.
This is fully autonomous
driving.
And it's been in all sorts of
different road situations.
So we drive in suburban
streets.
We drive in hilly areas.
We drive at night.
We drive during the day.
We stop for baby carriages.
We stop for red lights.
We hopefully don't stop for
too many green lights.
We drive through toll booths,
highways, bridges, congested
traffic areas, heavily congested
pedestrian areas,
and even down Lombard Street in
San Francisco, which is the
last clip here.
Now, in a little more detail,
the entire system can be
broken down into just
a few key steps.
The first step is to get an idea
of where the vehicle is
in the world.
So to do this, we first get a
rough estimate using GPS and
some inertial sensors that we
have on the vehicle, such as
the speed of each wheel.
Now, this tells us which road
we're on and maybe what the
closest intersection is.
But it doesn't tell us exactly
where we are in
our lane, for instance.
So we need to improve
this estimate.
And we do this by using Maps.
So the idea here is that we take
the laser that's on the
vehicle, and we look at what it
can see around it, and we
compare that to what
we have on the map.
And by doing that comparison,
we can figure out where in
that map we are, based
on what we can see.
And this tells us
very accurately
where the vehicle is.
So we know where it is relative
to its lane, where it
is relative to crosswalks.
And once we have this accurate
position, we can then overlay
some of this key information
about what the world is like
and what the vehicle
should do.
So we can add in where the
lines are, where the
boundaries are, where crosswalks
are, where
intersections are, where the
nearest Taco Bell is, all of
the really pertinent
information
that it might need.
And we can then augment this
with dynamic information about
what the vehicle is seeing using
its onboard sensors.
So this is all of the other
vehicles that are in the
vicinity, pedestrians, traffic
lights, stop signs, and so on.
Given all of that information,
we then filter it down, as I
mentioned before, into just the
key components that are
important for making the
decisions of where to drive
the vehicle.
And we then come up with a
trajectory for where we would
like it to go.
And this consists of a path
through the world, along with
a desired speed at each
point along that path.
And this takes into account
things like slowing down for a
vehicle in front of us, stopping
at a stop sign, going
through a green or red
light, and so on.
And once we have this
trajectory, the speed profile
with the position estimate, we
then feed this to the vehicle
and have it executed.
And we then repeat this
entire process
several times a second.
And the resulting system looks
something like this.
So on the left here, you have a
video taken from an onboard
camera looking out
into the world.
And here you can see this
run was done at night.
And on the right, you have an
internal representation of
what the vehicle is seeing in
the world and what it's
reasoning about.
So here you can see it
has static obstacles.
It has its map.
It's detecting other vehicles.
Here the boxes represent
other dynamic vehicles
that are on the road.
It's also reasoning about
traffic lights and
intersections.
Here, we started out in a
suburban area, and we whipped
onto a highway.
We then zip along the highway
for a while and pop back off.
So this all looks pretty
good, right?
We have a vehicle that has
driven hundreds of thousands
of kilometers in all sorts
of different conditions.
Seems like we've sort of
got stuff figured out.
Maybe we're done.
Maybe it's time to unleash
it on the world?
Well, not quite.
Unfortunately, when you're
operating in real road
scenarios there are a huge
number of special cases or
weird anomalies that you might
need to deal with.
One such group of anomalies are
interesting vehicle shapes.
And when you're first putting
together a vehicle detection
system, you might not anticipate
that you're going
to have to encounter cars that
are shaped like hot dogs.
But if you drive enough miles in
enough places, particularly
in the US, you will see all
sorts of interesting things.
You also need to deal with very
erratic behavior that
people in vehicles or
pedestrians on the street may
exhibit from time to time,
perhaps after jumping off one
of these vehicles on the
bottom middle here.
As well as the complex vehicles
that you might need
to deal with, there are also
road scenarios that we might
need to deal with.
Now, we love using Maps
to improve the
performance of our system.
But the road can change based
on construction or
re-painting.
And we need to make sure that
we're able to detect these
situations and respond
to them safely.
We also have to deal with
dynamic road situations, such
as accidents or emergency
vehicles that
may come in our path.
And then finally, weather can
present quite a number of
challenges for us.
If the road is entirely covered
in snow, if our
sensors feel the view is
occluded by rain or heavy fog,
we have a number of additional
things that we need to solve
as part of this overall
problem.
And then finally, we have
extra-special situations, like
this wonderful fellow here.
Basically developing a system--
[LAUGHTER]
DAVE FERGUSON: --developing
a system that's robust to
anything that the world
can throw at it is
really, really hard.
But again, this is
the good stuff.
It needs to continue to be hard
to make sure that people
are still interested
in working on it.
And we are still very
much interested.
So what's next?
What's next for the project?
What's next for driverless
cars in the world?
Well, let's take a look.
[VIDEO PLAYBACK]
-Hands-free driving.
Cars that park themselves.
An unmanned car driven by
a search engine company.
We've seen that movie.
It ends with robots harvesting
our bodies for energy.
This is the all-new 2011 Dodge
Charger, leader of the human
resistance.
[END VIDEO PLAYBACK]
DAVE FERGUSON: So I love
that commercial.
And you know, Google does have
a lot of power-hungry data
centers that could
use the energy.
But actually, we're more
interested in a future that
looks something like this.
[VIDEO PLAYBACK]
-OK.
-Well, Steve.
-OK.
-Off we go.
[CAR BEEPS]
-Auto-driving.
-Here we go.
-Away we go.
-Look, ma, no hands!
-No hands anywhere.
-No hands, no feet.
-No hands, no feet,
no nothing.
-I love it!
So we're here at
the stop sign.
-Yep.
-Car's using the radars and
laser to check and make sure
there's nothing coming
either way.
-I find myself looking.
-Old habits die hard, man.
-Hey, they don't die.
Hey, anybody up for a taco?
-Yeah, yeah.
What do you want to
do today, Steve?
-I'm all for Taco
Bell, myself.
-All right.
Well, let's go get a taco
at the drive-through.
-Now we're turning into
the parking lot.
-Yeah.
-How neat!
-There we go.
Now we kind of creep
along here.
Does anybody have any money?
-I've got money.
-No, I've got my wallet
right here.
If you roll down your window
and order a burrito.
Yeah, push that.
-How are you doing today?
-I'm doing very well.
How are you today?
-Good.
Thank you.
-This is some of the best
driving I've ever done.
95% of my vision is gone.
I'm well past legally blind.
You lose your timing in life.
Everything takes you
much longer.
There are some places
that you cannot go.
There are some things that
you really cannot do.
Where this would change my
life is to give me the
independence and the flexibility
to go the places I
both want to go and
need to go when I
need to do those things.
You guys get out.
I've got places I have to go.
-Yeah.
Bye now.
-Hey, it's been nice,
you know.
It's been nice.
[END VIDEO PLAYBACK]
DAVE FERGUSON: Most of you
here today live in the
developed world.
You and I have already won
life's lottery and been given
the opportunities
that we have.
What are we each going to do
with our winning ticket?
And you might think that
driverless cars or internet
from balloons are kind
of silly ideas.
And that's fine.
But what is it that
impassions you?
What might you push forward
in the world?
Can New Zealand be the
first country to use
100% renewable power?
Can we solve the diabetes
or obesity epidemic?
Google X and Solve for X is all
about trying to take on
these big, massive challenges
and try to make an impact.
There are so many huge problems
in the world and so
many that could benefit from
some Number 8 wire ingenuity.
So you tell me, what is next?
Thank you very much
for your time.
[APPLAUSE]
