Thanks for the introduction.
Hi, everyone.
My name is Patrick Perea.
And I work at the University
of Grenoble in France.
And in this
presentation, I will talk
about navigation and guidance
in augmented reality.
OK.
So this is what the
user looks like when
trying to look for digital
objects in his environment.
He has no guidance
information on screen
and no knowledge about
the offscreen environment.
Because of the restricted
field of view of the camera,
he has to turn around the spot
to explore his environment.
And if we include guidance
information on screen,
it must be on the
edges of the screen,
because the center is used to
access this digital information
that we call the
points of interest.
And finally, there might be
a lot of offscreen objects
that corresponds to
the user interest.
So to make a proper guidance
system in augmented reality,
we have two requirements.
First, the guidance
system should not
be intrusive on screen.
And secondly, the
guidance system
should be able to manage
dense environments.
So our final question is how
to visualize offscreen objects
in augmented reality without
overloading the user interface.
There are many
techniques that have
been developed to show the
location of offscreen objects.
We are going to detail
some of them here.
Obviously, we have the arrows.
So the arrows are often
used to show location,
to show directions to
offscreen locations.
However, they have never been
tested in dense environment.
Secondly, we have the Halo.
So the Halo is a
technique that shows
the location and distance
of offscreen objects using
circles.
So how does it work?
We simply draw a circle
here whose center
would be the offscreen
object and the radius
is the distance between
the offscreen object
and the edge of the screen.
So what the user sees on
screen is just an arc,
and this arc is enough for him
to mentally complete the figure
and understand where is
the point of interest
and how far is the
point of interest.
So the bigger the circle, the
further the point of interest.
The problem is that when
you have many, many objects
around you, it leads to
many overlaps on screen,
which really decreases the
readability of the user
interface.
So to resolve this issue,
the wedge technique
has been introduced
and it uses triangles
instead of the circles.
Again, the problem is
that when you increase
the number of points of
interest up to 30, well,
the wedge system is
not robust enough
to prevent the overlapping
problem to occur.
And in augmented reality,
we have the arrow plus.
The arrow plus is a technique
that works in 360 degrees.
So we display small points
on the edges of the screen
to show the location
of offscreen object.
The main benefit feature is that
it considers all the directions
around you, but the
problem is that when
you have many objects, it leads
to a dense cloud of points
in a small section
of the screen, which
decreases the readability
of the user interface.
If I summarize the related
work so far, we get this table.
So on the columns, we
have all the techniques.
On the lines, we
have the requirements
needed to make a
proper guidance system.
As you can see, none of
the technique is perfect.
None of the technique is fully
satisfies all the requirements.
So to satisfy the
requirements, we
introduced Halo 3D
as the technique that
works in augmented
reality that deals
with the dense
environment and that
is not intrusive on screen.
So how does it work?
It is simply an adaptation of
the Halo in augmented reality.
So we project 3D point of
interest on the device screen's
plane, and we apply the same
algorithm as the classic Halo.
And then to avoid the
overlapping problem,
we simply aggregate the
circles when they visually
overlap on screen.
So we take the centroids
of the projections
of the point of interest, and
we draw a circle whose center
will be this centroid.
And this circle replace the
overlapping circles of course.
So now, the purpose--
and it goes like this.
Yeah.
So here we have the
animation real time.
So now, the purpose of
our first user study
is to compare Halo 3D
with the arrow plus
because this is the
only technique that
has been tested in
dense environments
and in augmented reality.
And also to compare
Halo 3 with the arrows
because they are widely used
in today's navigation systems.
So we compare the
three techniques,
and the objective
here of the task
was to find an offscreen
object placed around users.
So the target was designated
by the green color,
and we asked user to
search for this target
and select it to
validate the task.
OK.
So we tested two
density of environments,
low density with five
points of interest
and a high density with
50 points of interest,
and two movement amplitude,
a narrow movement
inferior to 80 degrees
and a large movement
superior to 120 degrees.
So large movement is
basically to reach a point
of interest placed behind you.
For each technique, users
perform six interaction
for both environments
and amplitude conditions.
So it's basically
24 pointing tasks
for each technique
and 72 pointing tasks
for the whole experimental task.
At the end of the session,
we conclude the session
with an interview to
collect user's feedback.
And we obviously measured the
time to complete the task.
However, we did not find a
strong effect of the density
environments on the
test completion time,
and we did not find a strong
effect of the techniques
on the task completion time.
However, here what you can see
is the task completion time
for all the techniques and
for both type of movements.
We observed that users
spent a bit more time
using the arrow plus, here in
red, than the other techniques.
And actually, the
task completion time
is made of two steps.
The first one is the
visualization interpretation
time, which is the time needed
to look at the visualization
and understand which
direction has to be followed.
And then the second step is
the offscreen guidance time,
which is the time
needed to bring back
the offscreen point of interest
back on your field of view.
So if we look at
the first step here,
you can see that statistically
user spends typically
more time trying to interpret
the arrow plus interface
than the other techniques.
And there are not so much time
differences between the arrows
and the Halo.
And this is due to the fact
that the arrow plus display
small points on screen, so it's
much more harder and difficult
to spot the green one,
which is the target,
compared to spotting a
green arrow or green circle
on screen.
And if we look at the second
step, the offscreen guidance
time, there is not so
much time differences
between the techniques.
So this means that the
task completion times
is hugely influenced by the
visualization interpretation
time.
What can we conclude from
this first user study?
First, Halo 3D has similar
time performances as the arrows
while being less
intrusive on screen.
And secondly, the Halo 3 is
faster than the arrow plus.
However, from the
conducted interview
at the end of the session,
users raised concerns
about the way the aggregation
was visualized on screen.
Because Halo 3 only
indicates the direction, yes,
towards the centroid
of a cluster.
So we lose the information
about the spatial distribution
of the point of interest
inside of a cluster.
So from our first
two requirements
at the beginning
of the presentation
to make a proper
guidance system,
we decided to include a
third requirement, which
is the spacial
distribution, the ability
to show how the
point of interest
are spatially distributed
inside of a cluster.
To satisfy this
requirement, we say,
hey, why don't we modify the way
the aggregation is visualized
on screen.
And we came up with
two new techniques.
The first one is
based on the idea
the small overlaps
between the circles
will not disturb the user.
So we authorized small
overlap between the circles
to avoid too much
aggregation and be
able to understand how
the points of interest
are spatially distributed.
And it goes like this.
So if I post a
video here, you will
see one overlap, which
means that we avoid too much
aggregation, and
you can understand
how the point of interests are
spatially distributed here.
The second technique
is based on curves.
So the idea here is to display
curves instead of circles
for the aggregation to
be able to understand
the visual extent
of the clusters,
and it goes like this.
So now, the purpose of
our second user study
is to compare these two new
techniques with the three
previous techniques we covered
in the first user study.
Here, the objective was to find
35 points of interests placed
on one face of a machine.
So to reproduce complex
industrial environments,
we took a picture of a
real production machine,
and we printed this
picture on the 3 meters
per 75 centimeters large panel.
The task was subdivided
into two sub tasks.
The first one was a locate task.
Here, the objective for the
user was to estimate and guess
the location of the
offscreen point of interest
only by looking at the
visualization on screen.
So we presented on
the computer a grid,
and we ask user to click
on the cells in which they
thought one or more points of
interest were located, here
in red.
The second sub task was
an exploration test.
So here, the objective was
to search and find one by one
all the point of interests
on the machine in the order
they wanted.
For these experiment, users
performed the training phase
and immediately performed
the experimental task
for each technique.
So we ask user to perform
the locate task and then
the exploration task.
And when there were only
10 points of interests
left on the machine,
we asked users
to perform all
again this process.
Again the locate task,
again the exploration task,
just to allow us to study
which technique better
improves the understanding
of the offscreen
environment in cases of
high density environments,
in case of a low
density environments.
We conclude the session
with an interview
to collect user's
feedback, and we
measure the time to
complete the task
as well as the number of
incorrect answers on the grid.
However, we did not find
strong effect of the techniques
on the exploration
time and on the number
of incorrect answers, but
we collected strong quality
feedback in this user study.
So we took the overall feedback
given during the experiment
as well as the answer
in the questionnaire
to create this table.
So you can see
here on the columns
the technique we
tested and on the lines
the three requirements needed
to make a proper guidance system
in augmented reality.
As you can see, none of
the techniques is perfect.
However, I want to get
back on three details here.
The first one is that there is
a problem with the curves when
all the points of
interests are distributed
around you, because it
leads to a single curve that
goes from one side of
the screen to the other,
so you end up with a
visualization that does not
provide any useful information
about the offscreen
environment, just like 30 hours
on screen, you'll be useless.
We have kind of the same
problem with Halo 3 star
because it leads to
many overlaps on screen,
which were judged as
visually disturbing by users.
And finally, users felt more
comfortable using the Halo 3
stars compared to the curves,
simply because with Halo 3 star
you can see the complete
arc here on screen.
So it's mentally easier
to complete the figure
and understand where is
the point of interest.
But with the curves, you never
see the complete arc on screen.
So it's mentally more difficult
to complete the figure
and understand where is
the point of interest.
So what can you conclude
from this user study?
First, all Halo based techniques
have similar performances,
but Halo 3 star and the
curves were judged as more
able to show and display
the spatial distribution
of the point of interest.
So in this presentation, I
presented three variations
of Halo techniques that
works in augmented reality.
We saw that Halo 3D had similar
time performances as the arrows
while being less
intrusive on screen
and is faster than
the arrow plus.
Secondly, we saw
that all Halo based
techniques had
similar performances,
but the curves and Halo
3 stars were just more
able to show the
spatial distribution
of the point of interest.
As none of these
techniques is perfect,
future work will focus on making
a technique that automatically
adapt to his environment,
like a hybrid technique.
And this concludes
my presentation.
Thanks a lot for your attention.
And I'll be glad to
answer any question.
[APPLAUSE]
Questions.
Hello.
Lauren Devonne
from LMU, Alminick.
My question was how
works a projection when
a point is exactly on
the back of the user
because a projection arrived
in theory on the screen.
So how do you indicate the
user virtual object is really
behind the user?
Well, actually, from
the visualization,
you can't know if the point of
interest is behind or in front.
The only way to understand if
the point of interest is behind
is by moving yourself and follow
the direction of the circle,
for example.
So as you move, your circle
will be smaller and smaller,
and you will reach the point of
interest progressively, which
is behind.
So the only way to understand
where is the point of interest
if it is behind is by moving
dynamically, not statically.
Questions.
Yes.
And then another question there.
All right.
Go ahead.
Please.
Thank you.
There were some trainings, you
had some trainings, that 3 Halo
and arrow one, you
trained the users, yeah?
Speak louder.
You trained users before using
arrow ones and Halo ones,
so did you consider
random training
or first you trained
them Halo, second
you trained them arrow ones?
For which user's study?
Anyone?
Anyone.
In the training ones.
Well, no.
In the second user study, users
performed the training phase
for one technique and
then immediately performed
the experimental task
for this technique, OK?
So for example, if they
started with the Halo,
they will train with the Halo
and then immediately perform
the experiment for this Halo.
And then we'll switch the
arrows, train for the arrows,
and perform the
experiment for the arrows.
OK.
So this is the
second user study.
In the first one, we randomized
the way which technique
will be tested in which order.
Won't it affect
the way that they
perform the speed of doing it?
Because when they do
the third learning,
they are adapted to the
way that they should do it.
I mean, maybe random
learning will be better,
for example, first Halo
for the first user.
I don't know.
For the second user, arrow
ones could be the first one.
Yeah.
I think that could be
that better training.
Did you get what I mean?
No.
So why didn't you
randomize the training
on the second user study?
That's what she's asking.
If I do--
Why did you not
randomize the training?
In the second user study?
Yes.
Because there were
too many techniques.
There were five techniques
in the second user study.
So the problem to test all the
techniques in the same training
session and to do
the random order
is that if you started
by understanding testing,
for example, the Halo,
then you test the Halo.
Then you will
testing the arrows,
you will testing all the
techniques in the same training
session.
And then we'll perform the
experiment for one technique.
So maybe we thought that you
will be forgetting what you
learnt in the training phase
because there were too many
techniques.
So that's why we
did one technique
and for this technique, training
and then experiment, and not
random.
I'm not sure.
Yeah.
You can take it offline.
Hi.
[INAUDIBLE] University.
I was just wondering,
in your points
of interest all had uniform
importance of priority,
I wonder--
Yes.
How would you extend
your techniques
if you needed to show
different levels of priority,
as I said with those
points of interest?
OK.
So here in this
case, we consider
that all the points of interest
had the same level, OK?
The thing is that
in the application,
in total application, before
showing the point of interest
on your screen, you first
perform a filtering face, OK?
So you select the
point of interest
which you're interested.
For example, if you're in
the industrial environment,
you will select the
point of interest
that are high priority one, for
example, arrows on the machine,
alert, something like that.
You'll be filtering all
these points of interest,
and you will be
showing only the point
of interest for this priority.
So at any time right
now, we are showing
the same levels of priority
for this point of interest.
The thing is that we
thought about showing
various types of priorities
in the same user interface.
So we thought by
showing, for example,
distinguishing the levels of
priority just using the colors,
for example.
OK.
The thing is that we
thought that the colors will
be too much visual
disturbing for user.
Because, for example,
with a circle,
you have the radius
of the circle.
You have to understand
the shape of the circle
and then there is
the aggregation.
So there are three things.
And then you add the
color for distinguish
the types of priority
of points of interest.
So I think it's too many
things for a user for a system
that only you will be
using from time to time.
