- Okay, good afternoon.
I am Theo Amanatidis and I'm a PhD student
at the University of Cambridge,
and today I will try to
do my best and present
this thesis paper, instead
of the main author,
which is Charlie Hewett
and the paper is on
Assessing the Public Perception
of Self-Driving Cars,
the autonomous vehicle acceptance model.
First of all, I'd like
to start by thanking
the previous presentation
because it was very interesting.
We kind of have some similar topics
and I'll try to point
it out as we go along
in this presentation.
So, as indeed mentioned in
the previous presentation,
vehicles with limited levels of autonomy
are already coming up on the market.
They already are on the market
and deployed in different
places in the world,
in different environments.
However, while there's
clearly people who are buying
these vehicles, there's
little evidence to suggest
that users in general, kind of broadly,
desire the wide-spread use
of autonomous vehicles.
I mean, what I'm trying to say is,
we don't know if this is just
a specific subset of users
that are choosing to buy these vehicles,
or if this is a broader
trend in the industry.
More specific than that,
most existing studies do not,
of public perception
of autonomous vehicles,
do not use established user
acceptance models, okay?
And therefore, it might be
difficult to make comparisons
or interpret the results
between these studies.
For example, as this chart shows,
as the level of autonomy
increases basically,
and the level of interaction
between the user and the vehicle reduces,
it is unclear exactly what part
of that interaction is reducing.
Is this, as the previous
presentation meant,
is it the hands that are
taking off the wheel?
Is it that you're not paying attention
to auditory interactions
within the vehicle and the user
or is it eyes and brain that are off?
So what we're trying to do, is basically,
what this paper tries to do,
is establish a model of,
based on user or the previous
user acceptance models,
that allows us evaluation and comparison
between different studies.
Okay?
So our proposed approach
is to unify current efforts
in measuring acceptance
of autonomous vehicles,
to combine the different elements
between current technology
acceptance models,
car acceptance models, and
introduce our own element
of autonomous vehicles,
and levels of autonomy.
Furthermore, we believe that
the concrete industry-set
levels of autonomy
might not be intelligible
to the average user,
or might not be extremely indicative
of what the vehicle can and cannot do.
So, we want to enable participants
to better visualize these
hypothetical technologies
and how they are implemented
in their vehicle.
So as part of that effort,
this paper presents the AVAM,
which is basically the initials of our
autonomous vehicle acceptance model,
as presented in the beginning.
So the AVAM is a 26 item questionnaire,
which is basically an adaptation
of two previous models,
specifically the Universal Technology
of Usage and Acceptance of Technology,
and the Car Technology Acceptance Model.
And I'm gonna explain later,
specifically what parts
we have adapted and what
parts we've added in.
It uses six autonomy scenarios,
so these represent as zero
to five level of autonomy,
but they're not labeled level one,
or level three, or level five.
Instead, it's an explanation
that's given to the user
of each level of autonomy.
Finally, this model was
first piloted with 54 people,
and then developed with 187 participants,
all taken through Amazon Mechanical Turk.
Just a little bit about
their characteristics,
the average age is 22 to 65 years old,
with a mean of 34 and a
standard deviation of 9.2.
We have 111 male and 76 female,
sorry, not quite 50-50, but getting there,
and all of them have
been taken from the US
and I will describe a
little bit at the end
what this means in terms of the sample.
And finally, and this is
kind of reporting to us,
the full dataset is
available upon request,
so everybody who wants to use it
or include it in their research.
Okay, so the questionnaire was established
based on, as I said,
the two previous models.
It consists of some background questions
to understand about what our users are,
what the usage of the vehicle is,
do you have any previous
driving experience?
Actually, 181 of the 187 people
did have a driving
license in world driving.
So, the individual items,
we're taking to correspondents
as much as possible to UTAUT.
That means that out of the eight factors,
if all of the eight factors
of your total were included,
and then the ninth item was
the last factor of CTAM,
which is about perceived safety.
And again, I'll explain that in detail.
We kept also the seven-point
Likert scale, as per UTAUT,
and the five-point sLikert
scale was introduced
for the additional questions.
So the additional questions,
what are they are?
Well, as given a little
bit in the previous talk,
perhaps it is uncertain
for users what elements,
what level of attention
do they need to have when they're driving?
Do they need to use their hands?
Do they need to use their feet?
Do they need to use their eyes?
So basically we added
three additional questions
at the end of the
questionnaire to evaluate
if users knew what part
of their body, in a way,
they needed to make
sure that they're using
during the driving at
this individual levels.
We use the term eyes
instead of brain or mind
because we thought that this might
imply more complex engagement.
So it's just feet, hands and eyes.
And also, I'd like to
mention that the participants
had to answer the questionnaire six times,
one per each driving scenario.
This was done in a row.
Now there's obviously trade-offs
and I think that was mentioned
in one of the questions beforehand.
At what stage do you answer the questions,
and if you answered
them in an order or not.
We chose to do this in order,
which might introduce
some sort of bias in that
but the trade-off, the
positive if you will,
is that it will basically allow users
to kind of visualize in increments,
in hypothetical increments,
the distance between where
we are in the present
in the technology, and where
we will be in the future.
So this is basically what
I was saying beforehand.
Of all these scenarios, there's one that
proved to be particularly difficult,
and that is level four.
The reason why level four
is particularly difficult
is because the definition of
level four in SAE standards
is very broad and can
be divided, if you will,
in different ways.
For consistency in this study, we used
a definition that follows
a specific type of roadway,
so basically an extension of level three.
And by the way, you know
the text at the bottom,
this is just kind of illustrative.
This is not the text we
had actually in the paper.
You can find that on the paper.
In terms of the results
and what we're aiming for,
we are aiming for high
internal consistency
and external validity obviously.
As you can see on the right-hand side,
we had scores of 0.8 on all of the,
so this is the Cronbach's
alpha, I should point out,
on all the different factors
that are constituents of the AVAM,
and above 0.9 on some of them.
In terms of external validity,
we wanted to compare this
study with Rodel et al 2014.
However there are
limitations because obviously
the study at the time used
a different kind of scale,
didn't use SAE,
there were some differences
between the questions.
So we tried to match
that as good as possible.
What ended up happening is that
we had to exclude level three
because there was not, so
the Rodel et al did not have
the equivalent of what we now
call SAE level three in 2014.
Again, for external validity,
we couldn't quite fit it all
in this very short presentation,
so you can find that
in the paper if needed.
So having established this model
we went on to kind of
evaluate some questions that,
what might we use this model four?
Specifically, the first tried
of these two you can see.
How does the user acceptance vary
with the level of autonomy of the vehicle?
And which methods of control
do users expect to have
for vehicles with different
levels of autonomy?
So this is what I was saying
about hands, feet and eyes.
So what you can see here,
and hopefully this is not too small,
is the result of the nine factors
that were described earlier for AVAM.
So I'm not gonna go through all of them
because otherwise that'll take forever,
but what we can see in these,
so by the way I should point out that
higher is better for
everything except anxiety,
which obviously lower is better.
So what we can see is that generally
a trend towards reduction of, say,
performance expectancy
or effort expectancy,
as we increase the level of autonomy.
We believe that this may be because of
lack of trust in users,
but also because of the
increase in having to
make a hypothetical guess about
what this technology
will be in the future.
So people cannot quite perceive
how this technology will be
and how basically it will affect
them when they're driving.
So this is particularly visible
in things like perceived safety,
while the vast majority
of studies, for instance,
recommend autonomy as a
means to reduce accidents,
it turns out that users mostly disagree,
and as the level of autonomy increases
perceived safety reduces.
We attribute that to uncertainty about
where the technology is
and how they perceive it,
how the users perceive it.
In terms of the second research question
this is also somewhat different.
In this case, what we see is
that the level of interaction
between level zero and four
remains relatively constant
for hands, feet and eyes
with perhaps a little
reduction in terms of hands
as we go along.
But then there's a
massive, or a sharp, drop
between level four and level five.
We believe that this is because users are
understanding the difference
between level five
and all the other levels.
However it seems like
the difference between
level one and level four remains uncertain
in the eyes of a lot of the users.
So this is basically what
we wanted to highlight,
one of the things I wanted
to highlight, with AVAM.
So I have to go a bit
faster to conclude here.
So partial autonomy is
perceived to require
uniformly higher driver
engagement than full autonomy.
So this is clear from the last few graphs.
The public is not yet fully
convinced about autonomous cars,
and they have some
uncertainty, lack of trust,
and they're anxious about
some of these technologies.
Also, perceived user engagement
does not escalate clearly.
Some engagement may be cumbersome,
regardless of how often it is required.
So the trait we talked about,
so we heard about handovers
and taker of this previously,
this is an example of that.
However, there's a lot
more work to be done.
Analyses by age, gender,
driving experience
are yet to be concluded and hopefully
we will do that in the near future.
In addition to that, in this study
we only used US participants.
I think it would be interesting
to see the difference
between users from different countries
and see if there's a
cultural difference involved.
So in conclusion, this was
a standardized evaluation
of autonomous vehicles.
Sorry, AVAM provides a
standardized evaluation
of autonomous vehicles across studies,
and a meaningful assessment of
changes in perception over time
because you can re-do the study
over a few years to see
if there's a difference.
Similarly with different
autonomous vehicle technology,
so different levels of automation.
It provides high internal consistency
and external validity,
and the results of using
that to research questions
was that participants
generally are positive about
autonomous vehicles but with preference
to lower levels of autonomy.
The utility of autonomous vehicles
seems to be more
appreciated in level five.
And we hope that,
and this is kind of extension
to the rest of the room,
that findings should inform efforts
in simplifying autonomous
vehicle human-machine interfaces
across different autonomy levels.
Yeah, that's it.
I'm Theo Amanatidis, thank you very much.
(applause)
Any questions?
- Actually I will try pass
the mic around the room.
So do you have questions?
Raise your hand, anyone?
Oh yes, here we go.
Say your name and your
affiliation and question.
- [Audience Member] Hi,
thanks for the talk.
My name is Karl, from LMU Munich,
and I would like to ask about level four
and the presentation.
Before we've seen that there
is no takeover request,
so if there's no takeover request,
can you elaborate the difference to
level five and level three?
Thanks.
- Thanks for the question.
So I think this depends on
your definition of level four.
In some definitions of level four,
there's no takeover request.
So for instance if this is
in transit vehicles we've
only used within low speeds,
say, outside of normal roads
or if it's used within
kind of a certain very,
not necessarily very but
somewhat, constricted way.
So for instance, it
could anything weather,
it can be roads, it can be speeds.
In this case we only looked at roads,
and an example of that
would be, for instance,
that this vehicle is fully autonomous
in all city environments, OK?
But it's not autonomous
in highway environments.
According to the SAE
levels, that is level four.
There's no level three.
There's no level five either.
But you can apply the
same thing, for instance,
autonomous vehicle that's fully autonomous
in any situation except, say, snow,
and then has to manually
driven in the snow.
Or can be fully autonomous
up to 80 kilometers per hour
but then has to be manually driven
after 80 kilometers per hour.
So there will be some sort of takeover
between autonomous and manual.
So that's half the answer.
The other half of the
answer is that, clearly,
we are discussing this here
and we like to think that we are experts
in autonomous vehicles, right?
People who are not experts
in autonomous vehicles
do not know the difference,
do no know when,
if they buy an autonomous
level four autonomous vehicle,
do not know its capabilities
and when they can use it or not.
And I think that's perhaps
what we're trying to
kind of explain in the
second half of the research,
the second question that we had.
- [Audience Member] Hi, my question is
if you already have everything on US,
I think it would be interesting to see
where the people are in the US
who have answered the question,
because I think for
instance in Silicon Valley
there's driving around a
lot of autonomous vehicles
so it might be the scales
for anxiousness or anxiety
would be potentially different,
that would be something
that you could look into.
Whether the exposure
to those types of cars
will actually change anything to that.
- Yeah, thank you for that.
You're totally right.
I don't think we have
done detailed analysis
of which city people live,
but we definitely looked
at sort of the environment
that they lived in,
if there was urban environment,
if it was countryside, if it was suburbia,
so this kind of sort of level.
And we had a good representation of people
in urban environments and
suburban environments,
a bit lesser in rural areas,
but they were still represented.
So it's not as if all these
people are in Silicon Valley
but it's also not that these people
all live in Kansas or something,
it's kind of a combination.
I think you're right, there's
potential to look at a
even more detailed level, certainly.
- So we are right on time.
Thanks our speaker again.
