Hey everybody my name's Colin Simon I'm the product manager and if youAR we're a
firmly based startup really building on full
stacks we have software that that lets
you do everything from consume augmented 
reality content to create and post it
just just it's a full stack AR cloud
platform and in spirit its next event
which is kind of open access the same
subset I thought I would talk to you
guys today about what I call the hard
problems of AR and you are we've been at
it almost five years now we started at
the end of 2014 and along the way we
discovered quite a few of the heart
challenges that any kind of ensure
successful awesome
they are ecosystem is going to need to
have solve really like mature become
ubiquitous become something that people
are going to use as part of their
everyday lives
so one more thing I'll tell you is I
call this the heart problems of ARS kind
of a reference to cognitive philosopher
a guy named David Chalmers who wrote a
paper about heart problems of
consciousness where he talked about some
things about how the human mind
perceives the world is easy to
understand but some other things turn
out to be really far difficult to solve
problems and it turns out but teaching a
smartphone to understand how deep a
building is is kind of like that there's
no good solution to teaching a
smartphone just
see the world and so are the title and
reapplied it from cognitive science oka
augmented reality so without further ado
like I said that you are great company
you should check them out youar.io
At YouAR we've been really
focused on what is going to take to
build beautiful cinematic augmented
reality experiences stuff like what we
see in Ironman for instance where the
augmented reality just seamlessly blends
in with the environment around him, it knows
where the floor is it knows where his
body is animated content is never
passing through like a ghost going
through walls it just understands the
context and behaves the way we want it to.
That's a big challenge and there's a few
stumbling blocks in there. So let's get into it.
Right now is like the best time to
be working in Immersive Tech by the way, we have so much
hardware coming out right now
it's awesome how many different headsets
how many different virtual reality
pieces of equipment are out  there, it's kind of like the
Cambrian explosion of augmented reality
I mean just look at how many companies
there are right now solving some little
segments of augmented reality for
different domains it is truly the Golden
Age of augmented reality we're just at
the start and we're going to need to agree on
some things at the start I think and if
we can solve these problems in a
collective way it's going to be really good
So we're totally not scientists
over at UA our our our ownership comes
from the world of animation and I think
that's given them the right philosophy
and it's led me and other members of the
team really experiment and go out and
see the best possible solutions see what
fails see what works and lets them some
really good lessons along the way
let's lie a bit now so dizzy capital is
one of the VC groups that really looks
at the AR sector and sees what's going
on there
they're saying right now the install
basic people either and they are already
smart phone smart glasses is growing
it's gonna be huge huge revenue
opportunity just in the next five years
since the only get bigger from there
with that in mind we want to be on top
of that but there's a lot of versiv tech
is ready to change with just a few
cabinets how are we gonna get from here
to something like this it's going to
take working out some of these tiny
little kinks where right now I do go on
my phone the Pokemon just hangs out in
the middle of my phone's viewfinder no
matter where I pointed and I've seen
updates coming out that's supposed to
address that I'm looking forward to but
we gotta do it on a massive scale so I
present you the heart problems of air
these four domains if we can solve these
collectively not not just yet you are
but every company that's working on
we can share solutions to get over these
four problems we've got a really bright
future so let's get into it
problem number one the wall card so
augmented reality I think this is kind
of intuitive works best when it's a
shared thing when me with my iPhone and
my friend with their Android device and
a third person alone hololens
can all look at the same thing and see
the same experience not copies of the
same experience not like pokemon though
i don't want to go and catch the same if
you could shoe that you just find I
don't want to compete with you over one
big issue that's not really possible
right now in in limited contexts with
particular apps on the same platform
baby but but right now two devices can't
view the same they are experience the
way that my iconic your Android can view
the same website and that's that's a big
problem
similarly it doesn't end it devices it
goes into different operating systems
and really we're talking about they are
here but but this is the immersive
reality you know and ARS is one end of
the spectrum that goes from VR to an AR
and everything in between I think the
best AR experiences when people to
experience either locally or remotely
and so long-term the RNA
seamless clean way and that's not
happening right now in fact quite the
opposite look at this giant let's Apple
Facebook Microsoft Amazon Google Samsung
10-cent Biden Alibaba and so many others
are all building their own Eric's on
platforms and for the most part that use
their own proprietary formats for a 3d
content for broadcasting events it's
it's wonderful if you're fully invested
in Amazon but if you're like me and
you've got an Android tablet and an
Apple phone and in parentheses that were
writing different voices that's not a
good solution for you because you don't
want to be locked in by a particular
vendor and a particular solution
especially the end of this what if they
go away and place where my company
decided to place our air content that's
gonna be really frustrating I mean
ultimately its creators and consumers
that lose when one wall garden is so
when we look at the future they are
we're looking for solutions that that
really go beyond us and are going to
work for a lot of different people
different circumstances so that's
problem number one number two I call it
the consistent persistence problem so an
eight-hour works best when the real
virtual aspects all that seam work
together in order for me to like really
increasing experience I want to suspend
my disbelief when I see a flying
dinosaur come and land on the table I
don't want a question but it's real it
just just seemed natural
it should understand where the surface
of the table is lie down land cast a
shadow ultimately we need an air content
that understands it seen well enough
that our King occlude behind a building
when it needs to
it can collide when a building with
building what it needs to it needs to
understand the scene and behave the same
way the real physical object would not
see but that's this persistence issue
goes beyond that it's also important
that these experiences work from many
different perspectives so a lot of they
are at the moment is kind of locked to a
single angle where maybe me and my
friend can both share the same thing if
we're pointing our phones generally the
same direction but a lot of way our
murderer technology at the moment kind
of has a woman of maybe 20 if we're
lucky
30 to 40 degrees of freedom before
something breaks down and what person a
on one phone sees is diverging will be
different more and more and more from
one person DC is on their phone and so
we think that they are into really
compelling cinematic something that
believe and use in my daily life it has
to just work consistently for many
different perspectives so what I've seen
from my angle is what someone else can
see from a different visual perspective
from a different angle keep in mind that
phone might have a different camera lens
in the shape of that camera lens might
be violent set a bubble it literally
takes in the light from that scene
differently so there needs to be a
robust translation layer that
understands the scenery the pulse
translated so lots of different devices
to understand it and if that translation
is good enough it should work for a
virtual view our off-site - we should be
able to reconstruct that scene using map
data using the latest data from the
people who are on site doing something
in a our to facilitate a VR experience
that goes right along with it and I also
want to mention that persistence in this
spectrum persistence is a big buzzword
right now every company who's doing an
error cloud platform likes to talk about
how its persistent in the case of Google
they mean persistent for about three
days before their cloud anchors expire
and so content that you stick somewhere
and they are will last for a little bit
but it doesn't really last forever and
the permanence might be very static you
can place that object there but if
something about the scenery changes what
used to be a wall once that wall gets
knocked down that content will still be
floating right there in the air unless
it can understand that something about
the scene has changed and so for true
persistence it's not good enough to have
a static model of the space that model
needs to be updated all the time with
new information
patience changed over time it's a big
one I think really key part to to
building that palpable realistic
ubiquitous augmented reality that we've
all been sold in movies let's talk about
the shared canvas problem so the better
your device understands location the
better the experience you can facilitate
and if one device really good
understanding maybe it's shared with
weaker devices so we've got some we've
got a microsoft hololens of the office
it's really awesome we can take more
detailed information about of space than
what a smartphone can understand
so Viking pull that information and
share it I really get this song it's
greater than the first situation where
if we can kind of share information from
the strongest sensor and pass it down to
the sensor we get a lot closer to having
that shared scene understanding that I
was talking about earlier
however sharing is really hard a lot of
different devices a lot of different
platforms and they all kind of have
different languages in which they share
their internal representation of the
scene right and so it's no small taxi to
arrive upon I have a shared non
proprietary language for describing a
location so that a hollow line is a
magic lead an iPhone and an Android
device can all see and accurately share
the same location and see the same stuff
superimposed with the same submillimeter
accuracy and then move in a nice
synchronous beautiful fashion you gotta
remember the way the Internet is built
as much
visual connections it's like making
phone calls there's not really sort of a
multicast and so if you want an
explosion to happen at exactly the same
moment on a bunch of different devices
you have to find a really tight way to
synchronize that and that becomes a
tougher challenge the more and more
users you add and so they do it for four
or five users is not so difficult if I'm
doing it for a stadium of people try
getting 10,000 people on the boat ascend
and get them to all see the same
experience in synchrony it becomes a
really tough problem and in order to
have a truth shared canvas that world
painting on it has to be shared not only
location and shape but also over time we
have to be looking at the same thing at
the same time and the other thing about
any sort of crowdsource canvass that
that is pulling information from a lot
of different data sets and sharing your
hosts each other is privacy becomes a
big elephant in the room all of a sudden
like I'm going to scan my bedroom so
that they can put in our content all
over the place I want to put a virtual
alarm clock next to my bed so I can look
at it when I wake up in the morning I
don't necessarily want to invite
everybody in the world into my bedroom
so I need a way to share that part of
the canvas selectively with other
members of my household with close
friends who maybe I want to be able to
visit me
presently through the art but that
canvas needs to be distributed in such a
way that only the people who really want
to have access to a dude and then our
last problem I call the context
discovery problem and I'll just jump
ahead one slide real quick to show you
there's a really wonderful short video
on Vimeo made by really talented graphic
artist called hyper or yellow some of
you to survive at have seen before is a
screen cap from it it demonstrates a
potential augmented reality future just
completely inundated with everybody and
their long superimposing their digital
content all over the place and it's
really exciting and visually and that's
why context discovery is a really
important thing and so on the World Wide
Web you visit content I'm forgetting
about the real world that I'm in zeroing
in on the screen I am usually traveling
to these of semuc information but that
whole model gets turned on its head and
ever meant to reality now the
information is coming to me it's coming
in my context I know what we're going to
its context but that means that the way
I discover content has to be different
it needs to be context aware how do I
discover content that isn't near me
right now but still might be relevant
if dozens and dozens of different
authors have all placed content in Times
Square in New York City do I see a
little bit does my game engine and what
everybody sent using a look at this try
and render and everybody's 3d model
ovary over each other that would be
terrible
so there's got to be some sort of system
for determining what content is placed
where what content takes precedence
based on who I am and what my
preferences are kind of the permissions
maybe my friends who have subscribed to
that sort of thing but this system
hasn't been defined yet at least in
cochin shared ways there are individual
proprietary solutions out there but are
we gonna make this work for everybody
are we gonna do it in a certain
organized this way because we did what
we do need but the same way with a
website if I am on the web address is up
to me who is allowed to place content
under that domain name
we're going to need spatial domain when
we decide we can place what we're who
can see what we're who can alter what
we're it sounds really in all at first
but when you start breaking down the use
cases like if I have augmented reality
on the factory floor to help my workers
understand how to maintain a device or
build something on an assembly line
maybe that's a corporate secret and I
don't want that automated reality to be
visible to someone who's staring into
the side of the building
their glasses we've got to come up with
a robust provisioning system for that
that's good
security and permissions and sharing and
all of that baked right into it if
that's the work of a single company then
it's something better than that and
that's not a scenario much too much
enter the narrow side so this term is
newer than you are this term emerged on
the scene after we were already hard at
work for two years and what do people
mean when they say the Aero Club well at
its basis the air cloud is a shared soft
copy of the world it's optimized for a
are capable smart devices so we take at
the moment primarily point cloud data
and we merge all together into sort of a
large spatial index so if you go to a
location take those GPS coordinates okay
does anybody ever scan this location
with the smart device before and if they
have I can use that information to this
detector the internal representation
thing I was talking about before I can
use that information to inform my device
before it's ever gotten a scan the
location about what to expect it helps
it lock on to and understand the scene
so that I can do that after collision so
it can do that accurate occlusion so
that I can manage that real
in between 10,000 users so we take all
this spatial information about location
point clouds the geometry let's say at
the moment but as it grows and as our
bandwidth grows we're going to start
transitioning from sort of static
persistent maps to dynamic maps that are
always updating so what you see changes
over time so that's what may our title
is ideally and like I said it makes
possible this ability to localize your
device position stuff accurately
superimpose digital content accurately
really capture a lot of information
about the space here but it does come
with challenges who owns the air plan
right now lots of different tech
companies are working to build air kind
of like data sets that they're
incredibly exclusive and siloed at the
moment and I'm not interested in they
are cloud that Google gets that behind
closed doors I'm not interested in the
air cloud becoming the next social graph
something for Facebook to sell to
Cambridge a melodica it needs to be a
shared resource something that we can
all benefit from and that's going to
mean that a lot of companies have to
come together we have to agree to an
open decentralized framework for the way
we're going to represent the
relationship
or madam or by creating their own AR
format instead of just using the one
with the Cronus groups so kindly
proposed point clouds right now is a
White House format war where Google
Apple let's say you are sixty blue
vision ubiquity six a bunch of other
companies we're all coming up with
different ways to describe the geometry
of the location but you know what
ultimately if all devices can understand
a single language that's gonna benefit
us all a lot more much along the same
lines we need a shared framework for
privacy and regional compliance if I
want to I went to reality in Europe I'm
gonna be damn sure that I need to comply
with privacy laws that have come to
affect in Europe ensure that somebody
has the right to be forgotten
so just because I scanned my house and
did some a Harnett today doesn't mean
that when I move out of that house I
don't have the right to take that
information that if I'm in the EU
citizen and so we as companies who want
to do business and you need to be
mindful of that and figure out how we're
going to comply with the regional
Sanders are we going to respect the
privacy and dignity of individuals when
we were up there very personal
information for the purposes of doing
this awesome augmented reality stuff we
should do it in a way that's going to be
responsible or respectful and it's gonna
work for a lot of different people long
term but I do think that the air cloud
can solve these hearts problems that are
outlined for you but only if we come
together all the different companies
interested in working on these problems
right now
but we can solve the wealth government
only if we agree to a decentralized
index and we can solve consistent
persistence but we better have a common
standard for describing and we can't
have
but only if everybody I'm sorry share
canvas but only 30 contributes done
sheriff damages and we can have same
context discovery but we have to agree
to some rules about who can place
content where so with that in mind do
you a our is very proud to be one of the
founding members of the Greek Albany our
cloud is an emerging industry standards
body that is working on how can we make
a version of the open air club that
isn't going to be the proprietary data
set of Facebook or Google or some
company going to be a shared resource
indexing spatial data taken by many
different platforms in many different
contexts but at least it's a shared
jumping-off point where we can all
discover the same content in a way and
it's gonna be the same all of the
browser model I want to see the
immersive web not that dozen different
commercial products that are on private
words so Chase Jarvis software developer
when the iPhone came out in 2009 he
wrote one of the very first camera
applications that came out on the house
so we called it best camera and its
premise was simple the best camera is
the one you have with you doesn't matter
if you have a cool DSLR at home it's not
with you when when the cool event that
you wanted to capture happen and much
along the same lines I believe that the
best augmented reality is the one you
can access and that's why you are we're
so focused on reducing barriers of
cross-platform solutions they're going
to make sure that the immersive lab is
not the exclusive domain of one walled
garden but instead this thing shared
across many different devices platforms
location
I think it's gonna be great
