Hello everyone, thank you for joining.
Hope you're having a great
experience attending build
from the comfort of your home.
My name is Astrid and I'm a
senior business program manager
for Microsoft for startups team
today I'm joined by Aditi Sharma,
who is the program manager in
Azure Autonomous driving team,
and Ben Lennon was a senior director at
Sinjen in the next 30 minutes or so.
You learn how autonomous driving
startup sinjen leverages Azure to
create a flexible architecture and
core universal driving capabilities
that can then be deployed.
From different vehicle types and target
different applications at this time,
however, I wanted to remind everyone
of our code of conduct at Microsoft.
We seek to create a respectful,
friendly,
fun and inclusive experience
for all of our participants.
We encourage everyone to assist us in
creating a welcoming and safe environment.
So let me talk a little bit about Microsoft
or startups program to begin with.
So Microsoft for startups
is an equity free program.
It's designed for C to series
CB to be startups.
Essentially we have two pillars designed
to help companies quickly scale.
The first pillar is about access
to technology,
which includes up to 120,000 United
States dollars of free Azure cloud
powerful developer tools like
Visual Studio and GitHub Enterprise.
In addition to Microsoft power platform.
And more importantly,
collaboration tools in today's Day
and age apparently like Office 365.
We also offer enterprise level
Technical Support and architectural
design sessions along with one on one
consultations with our engineering teams.
The second pillar,
really then is centered around
business acceleration.
Connecting innovative startups with
Fortune 50 hundred 500 customers
with a streamline path to partnership
where our start-up engagement manager
who is dedicated to helping startups
navigate their partnership at Microsoft.
In addition to that,
we also provide assistance in getting
your startup solution listed in
the commercial market place so it's
available for customers to download
or diploid where Azure marketplace or
Microsoft app source all around the world.
And most importantly,
connection to Microsoft Sellers
who are then compensated to sell
your solutions to our to their
enterprise customers.
Now,
attendees can definitely learn
a lot more about the program by
visiting startups.microsoft.com.
Having said that, for today's session.
I just want to remind everyone that
there is so much innovation that's
happening with machines and at
the pace at which it's happening,
it's unbelievable.
We at myself for startups have a
focus program on autonomous driving
because machine autonomy because
machine autonomous are autonomously
empowering the machines is really the future.
So with that let me hand it over
to Aditya to talk about automata,
autonomous driving program.
Thank you.
Thank you, start.
Uh so hi everyone, my name is.
I'm a program manager on the Azure
autonomous driving team and so I
just wanted to take just just five
minutes of your time here to basically
go over some of the key motivations
that led us to set this program up.
So as we all know, autonomous driving
is a complex problem to solve.
Some would even argue with the most
complicated technological challenge
humanity is trying to solve today.
Now the challenge is compounded.
If you're if you're an
autonomous driving startup,
becausr being a startup,
it was too in a very unique kind of
high risk high reward situation where
on one hand you have an opportunity
to make a very fundamental impact
on how people live their lives.
Everything from just moving from point A
to point B to just infrastructure in cities,
to the immense potential
environmental impact you can have.
Now chasing after this this.
Uh, this almost next generation technology.
It, on the other hand,
comes with the big challenge of
being high resource constraint,
and it's really the balance that you
have to strike between these two
that basically leads to the question
of how can we scale like bill bill,
bill,
trying to build this sophisticated
technology.
But we have this challenge of
scale and that kind of becomes
a chicken and egg problem.
Be'cause its very important in order
to get that to have infrastructure.
Uh, that is not only optimized for uh,
for for your workloads across
the lifecycle of your product,
but it's also very important to
optimize for cost and the challenge
we are seeing here is that.
Solving autonomy itself is is
the most complicated problem
like I just mentioned an we,
the startups working in this space have
some of the smartest minds on this planet
working toward that to solving that problem.
One thing that can become a challenge
as they try to scale though,
is also having to work on figuring out
how do we set up this infrastructure.
Cause we're talking about things
that that are unprecedented, right?
We're talking about doing
everything from ingesting.
In some cases,
multiple petabytes of data every
single day from a small fleet of cars,
right? To?
Then running large scale simulation jobs to
thousands of cores of GPUs for training jobs,
and then to these really high.
Really complicated hybrid type scenarios,
for example,
hardware in the loop testing
which is scaled by cloud.
So all of that requires a lot of
resources to be again put in,
so you're essentially putting in
more resources to save yourself more
resources so that you can scale and it
becomes a really complicated challenge,
and that's where we've that's where
that's where this program comes in.
What we realized working over the
last few years from everyone from
like a small early stage startup to
a large scale auto manufacturers.
And suppliers, uh,
we've seen all of these challenges.
We've not only learned these about these
challenges from our customers and partners,
we've we've experienced this these with them,
right?
So we've grown through
these challenges with them,
and that has led to Microsoft
building a lot of capability
and a lot of expertise around
those infrastructure challenges
specifically for autonomous driving.
An we've been getting a lot of really
good feedback from from our initial.
Early engagements in the start of
community that look we as much as we
as we love working with with Azure and
we love leveraging other Microsoft Tools.
What really sets Azure apart is
this access we can get to do these
X through these industry experts
to the solution experts where we
things that could potentially take
us months to solve can be solved
within days because you because these
these experts have done this so
many times with so many times with
other customers and partners right?
And that's, uh,
that's really what led us to
build this program.
So just quickly about the program,
because I do want to switch over to Ben
and not not spend too much time on this.
Basically,
you get the access you get access to
the premium tier of from Microsoft or
start up so you get all the benefits
that Astrologist just talked about.
All that awesomeness, right?
And on top of that,
you get 2 specific very specific sets
of acceleration packages, almost right.
So on the technical acceleration
side you get one on one axis,
you get one on one sessions with.
Uh,
you know our our cloud architects
are our engineering teams,
our product teams and that's
events be cause you're basically
getting direct access to people who
are building capabilities and features
which not only you're using right now,
but you can have a very direct
impact on shaping those capabilities
to meet your future needs.
And that's huge and we've gotten
some amazing feedback on that.
We also have in some cases some
really good code development
opportunities with our engineers and.
Access to those 80 capabilities
that we're building in it,
like before, they before they're
even released to the market.
Then on the business
acceleration side obvious,
I mean being being any company in
autonomous driving is a tough job, right?
So anything we can do to Oak to
leverage the immense sales cycle?
We have this the Salesforce we
have and we go to market programs
we have and all the networks and
connections we have within this space.
We aim to leverage all of those.
Find customer opportunities,
networking and as well as even
showcases in many example.
So I would love for you guys to go to
our website AKA Ms Slash 80 startup
to learn more about what we're doing.
But our goal really here is to empower
startups in the autonomous driving
space and make them more successful.
So with that I am going to hand over to
bed to talk about some of the Softy Softy,
most insane and awesome stuff
within the autonomous driving.
Space that you'll see.
So Ben,
what do you thank you?
So let's start off with a video here.
Quick video, so here what you see is
collection of some of our autonomous
vehicles from the last few years.
You see, in a variety of different settings
and this is a new project that we took on
just recently where after working with
more road based vehicles, sedans, SUVs,
utility vehicles like you see we we had
pilot where we decided to make a sweeper,
autonomous and industrial sweeper and.
The point of that video was not to show you.
I mean obviously prototypes
are not the end goal,
but prototypes are crucial to getting
wheels on the road and creating
high quality real world data.
And really,
data is what's behind the direction
in the approach that Sinjen is taking,
so we don't look at autonomy in
a vacuum as its own disruptor.
We see it as part of a bigger revolution.
That all hinges on Monday to write industry
4.0 is very closely tide into that.
An really pulls all these pieces together.
So for us being a more software
centric company,
we look at how can we leverage data?
How can we create new data and create new
insights so that autonomy doesn't aim to
displace or disrupt work as we know it?
It aims to improve to
create new productivity.
To to augment the human workforce to
make them safer and more efficient.
And that's really where where we see
our software working with the machines
in with the operations of our partners
to unlock new value and do it soon.
So where that data comes into
play is is in our software stack.
This is really our our bread and butter,
right? So the the product that we
put on our vehicles we call the core
the core technology stack drive MoD.
And this is a very simplified
version of that.
It would be way too much of an eye
chart if I were to try to show you
a proper block diagram an we would
need to sign NDA's with everybody,
which obviously is not feasible.
But uh, this this goes to convey that hey,
in this in this full full stack
autonomy software as it's called.
There's all these subsystems,
and within these subsystems there
are components and the way that you
architect this system plays such a
pivotal role in what you're ultimately
able to achieve with your autonomous systems,
and for us and for our customers.
The reason we show this simplified
version is to say, Hey,
we we're here where your providers,
your experts, and autonomy.
All you really need to worry
about is that if we work together
with our hardware partners.
With our vehicle partners
and get the right sensors,
create the right type of data from
the sensors for the application.
We have the software system that results
in telling the vehicle what to do so
that it can navigate asim safely so
that it can execute tasks that add
value to the operation and within this
we build this design intelligently
in a modular way so that when we
need to write code from the ground
up because we have a unique use case.
Or a solution for a particular
need doesn't exist.
We go ahead and do that,
but it also opens us up for
great partnership opportunities.
Even within our embedded driving stack,
let alone with in some of the
services that will talk about soon
with Microsoft and Azure that are
pivotal outside of the vehicle.
So let's delve into some of the most
exciting technology that's that's
in the autonomous driving space,
and where many of our brightest minds
are going and also helps to convey how does.
How does a machine even operate
itself autonomously, right?
So at the top here,
you might see something familiar
if you've been following autonomous
driving or computer vision at all,
so we have a camera image.
We have 3D bounding boxes that are
detecting an localising and classifying.
Pedestrians and vehicles in
this specific visualization.
And below that will we have a?
Lean vector based,
abstracted view of that world, right?
So up above.
Even though you're just seeing a camera,
the bounding boxes are actually being
generated by a sensor Fusion based system.
So we have multiple different
sensors in different sensor
modalities contributing here,
which is why you'll notice we have
object distances to each of these
objects which are very accurate,
which would otherwise be
relatively difficult to do.
And just a camera based system,
but this is the view that
helps humans to understand
best in terms of translating to
what the machine needs to see.
So then if you look in the bottom half,
if you imagine that you had not
seen that there was a street in
downtown Mountain View in the camera,
you could very well apply this abstracted.
What we call the world update view to
essentially any driving application, right?
If you boil down to these fundamentals.
Of. Recommended lanes that should
be followed that Macon textually
inform either the behavior of the
ego vehicle or of other actors in
the scene and our recommendations,
but help guide you in terms of the most
likely outcomes of what actors in the
scene will do and planned path so you
see that kind of green carpet that can
occur in whether you're in a mining
operation or you're in a warehouse,
you're going to try to get from Point A
to point B with your industrial machine.
And all of the actors in the scene, right?
If you think of them as as cuboids,
it helps to classify them.
You can assign profiles to the type
of classification that you have,
but in the end of the day,
you really want to know where is there
a thing and how is that thing moving.
How do I predict its motion so that I
the ego vehicle know how to respond to
dynamic seen elements in real time?
And that's where the here is a red
fence and now will jump over to
video so you can see this in action.
You'll see this adjusting dynamically.
Right, so here you can see the scene moving.
The fence is changing colors for a
variety of reasons right? You have.
You have inputs from the map that
might tell you hey,
this is an area where we always need to stop.
In this case it's across walk in a mine.
It might be around a blind corner
in a tunnel and you see those.
Those fence colors change dynamically
based on information it's getting
from actors in the scene as well
as their predicted paths come into
where the vehicle wants to navigate.
To complete its mission.
So with that level of abstraction,
that's a key element into how we re
purpose our software to different
types of use cases and will get a
little bit into how we do that.
With different vehicles.
An with fleet level learning here as well.
So most of what I just mentioned
was focused around drive model.
Like I said,
that's the that's the embedded
software that goes into the vehicle.
Now.
It doesn't exactly work that you say, OK,
I've got my sensor suite for this vehicle.
I've got my computers in my processors.
I've got the software stack that
enables it to drive autonomously,
and you go upload all of that to the vehicle,
install it on the vehicle and say,
go drive yourself autonomously,
go create value.
That's not how it works.
The there's really a symbiotic relationship
between that embedded solution
and the more enterprise level.
An data inside that can be extracted from
the solution that's out in the wild,
which is why we have fleet
management systems that range from
operational analytics to being able
to figure out real time diagnostics.
Update routes, see that the
vehicles are in good health,
and a software development kit
which includes a data pipeline.
How you get real world data out of
the vehicles into the cloud into
your servers so that you can train
your machine learning algorithms.
Uh, so that you can create insights,
cover more corner cases and all of
this really has to work together in
order to create the type of value
that industry 4.0 promises, right?
That that data and that data driven
approach that underpins unlocking new.
Business efficiencies,
and that's where Microsoft really
comes into play,
so this is my most offensive I chart,
so let's have a quick look at it,
and then I'll simplify it a bit
to land the key takeaway.
So on the left side we have the pipeline
as it is at our deployment site,
so where we would deploy with a
customer on the right is the more
development focused side of the architecture,
which for the most part runs through.
Our headquarters in the Silicon Valley,
so that's why you see some uniqueness
there on the right with our
development team or IT department
in doing much of the configuration.
But if we ignore that.
You can see that our deployment site and
our development site are very similar,
right?
These very closely mirror one another
with the main difference being that
we haven't really had the need.
We don't think that it's it's the
right approach necessarily to do
on Prem storage and deep learning
training at our customer facilities,
so we do have that option in our own sites,
in our headquarters and that's.
The main difference that you see here
and where Azure has been pivotal in
in this entire set up this pipeline
that you see here is enabling us to
seamlessly get data off of our vehicles,
even if,
like like was mentioned before,
even if it's terabytes of data,
petabytes of data,
we have a method by which we
can physically take solid state
drives off of the vehicles,
upload them into into our drive racks,
then we use the Azure stack.
Edge and gateway solutions to either.
Curate that data and send it up to our
cloud storage or on the development
on our headquarters side to do
machine learning in the actual Azure
stack modules for smaller workloads
because sometimes it doesn't make
sense to push it up to the cloud.
Depending on the workload,
size and this whole this whole architecture.
This is what let's us get data from
from our vehicles that are driving
near our office data from the fleet
that is deployed with our customers.
And get fleet level learning
and sharing across our entire
product offering so that we can use
data from one corner case that occurs
on the other side of the world to
inform and define the new releases.
The next versions that we push to the drive
MoD updates that improve the performance
of the vehicles out in the field.
And it also let's us test
in a safe environment.
In our development facility and
have testing methodology that's
indistinguishable from Aurora testing
ground that's indistinguishable
from the deployment ground.
So we know once we validated in our
development and testing facility that
we have essentially a one to one.
Situation where that can now be adopted
out in the field and with all that
share data and the the offering being
up in the cloud and backed by Azure.
As it is,
we can then offer interfaces to our
customers via via dashboards and let
them filter through enormous amounts
of data and in a more user friendly
way so that not only are we gaining
value from being able to drive our
autonomous product development with data.
We're also letting our customers unlock
data about their operations that they
may never have had access to before.
And and that that whole integrated data,
software architecture and and our
decision to really be a software centric
company and not tie ourselves to a
vertically integrated vehicle platform
is really part of what sets us apart.
So by partnering with hardware manufacturers
by being intelligent about for instance
creating hardware modules that might
reduce the amount of calibration that's
needed by creating a building block with
multiple sensors mounted together so that.
Whenever you you can see the wings here
on some of these vehicles in our fleet,
so that whenever you move
to a different vehicle,
you're not calibrating four or five
different sensors to one another.
Every time you have a single module
that's essentially pre calibrated,
an enables you to streamline that process,
and every time we bring up a new vehicle,
every time we do a new calibration,
we essentially grow the body of our
core technology so that the next one.
Becomes easier and the next one becomes
easier and we now the spectrum here is
growing from small machines like you saw
the the sweeper that I showed the video
earlier to these 15 person buses and
heavy machinery that where that we're
scoping out with some of our partners.
And. Finally, the last piece
of that puzzle that makes that.
With us, uh is a little bit
different compared to what you might
typically think of with autonomy,
which again positions us in a unique way,
is that the things that I
shared with you up until now?
I mean, these are a lot of the
best practices that if you've
been following along in the news,
you'll see some of the best names out there.
The leaders in autonomy that have
billions of dollars of funding
mean they would tell you similar
stories to what I what I just
went over the last 10 minutes.
The difference is that these the
robo taxi long haul trucking,
the highway automation markets
that are mostly more consumer
focused and these are the big
multi trillion dollar disruptions.
They tend to be kind of
an echo chamber right?
So you have a few of the leaders
in the space.
They often target more than one
of these talent goes from one of
the leaders to the other and you
end up seeing a lot of a lot of
similar solution that's pulling the
technology pushing the technology.
Forward in all of these.
All of these verticals in an exciting way,
and we were seeing commercial
and industrial applications.
Kind of get left behind.
So what we do is we take the best
design practices and we're bringing
it over to an industries that can
leverage that technology now that might
have more constrained environments.
So when we go and we develop and we
train our models based on urban driving
in downtown Mountain View in San Francisco,
like some of the robo taxi folks do.
We're in we're ahead of the curve
when we then go to an industrial site
that has an order of magnitude lower
actor density and slower speeds,
and that's really what's letting
us create value now.
An shorten that time to ROI.
That's a concern for some of the
very far reaching autonomy verticals,
and this is just putting numbers to that,
so we won't belabor this,
but it's to say hey,
even the quote Unquote smaller autonomous.
Pickles are huge and the road to to
ubiquitous autonomy will be a very long Rd,
so we,
for efficiency of our resources,
are focusing on these industrial and
commercial markets were creating
value with our customers today
and we see a huge potential to
grow into down the road.
And with that please please
come learn more about us.
Check us out and will turn
it over for some questions.
Thank you Pam. This is great.
The I am actually is buzzing with
with comments and questions.
Really sort of the theme that
we're seeing is around two things
and it would be great if you can
comment a little bit about it,
which is what parts of technology
or embedded and what parts and what
requires connectivity, sure, so.
Adam at a very broad level.
If it's safety critical,
it's probably embedded,
so if it needs to happen in real
time if it could impact decisions
that the vehicle will make in terms
of the maneuvers it takes and its
interpretation of the scene around it.
Those are embedded solutions.
I mean, we're talking millisecond level
latency's that need to be achieved there,
but beyond that I mean
there's great value in.
Opening up a lot of the other parts
of the data that is gathered, right?
So the fleet management.
Obviously we don't want that embedded right?
We want connectivity.
We want to be able in near real time to
tell someone who an operations manager
who's sitting in an office that hey,
there's an issue with this vehicle
or let them send a message to
the vehicle to say I need you to
take a different route than you
thought you were going to take.
So for the most part and less,
it's safety critical.
Anything that's dashboard analytics related?
I mean,
we try as much as we can to
leverage connectivity and push that.
Push that up to the cloud whenever possible.
And there's also flexibility there
because some of the some of the
operations that we have don't
have constant availability to
connect to to either 4G or Wi-Fi.
So we also have the capability
to queue those things and send
them up in larger packets later.
Talk about data and connectivity. We talk.
We immediately in today's Day and age.
Start thinking about AI in email.
Can you comment a little bit around?
Where is sinjen currently
using AI and Emil injection?
Yeah, so in general we are a startup.
Write a detailed, mentioned a little
bit about risks of and so in general.
Not only are we start up,
we are also working in these
safety critical applications,
so we want to derisk our
technology as much as we can.
While having leading edge capability,
so where we try to use A and ML is where
we know that it will create value,
not where it might.
Experimentally through some additional
R&D maybe create value in theory right?
So perception for instance computer vision.
There is no doubt that has been proven.
Deep learning outperforms non deep
learning methods, with that said.
There are still.
Concerns over determinism and
safety of what are still relatively
nascent solutions in deep learning,
especially for safety critical applications.
So where we do have deep learning
or a I wear it ties into against
safety critical systems,
we have a redundant deterministic path
which might be represented by more
deterministic calculations as opposed
to inference based predictions so that.
If there is some sort of uncertainty
or low confidence in the AI,
the higher granularity AI path we
have a fall back that can create
a fail safe method or we have
sensors that are very ruggedized,
very course,
but you know in a bind can help you to
ensure that you never come too close to
an object that a collision would occur,
so that's kind of our approach.
Our methodology use AI where it's
clearly going to create value
that you can't create without it.
Well, when you do,
be sure that there are fall back plans
because this is a safety first industry.
At this time I know where
were little overtime here.
I wanted to thank you, Ben.
An idea for joining us today.
Thank you again and thanks to all
the attendees were or tuning into
this session with that, it's a wrap.
Enjoy rest of the sessions and
build and hope you stay safe.
Thank you all.
Thanks everyone. Thank you.
