>> Hey friends. Azure
Cosmos DB is a great way
to store both structured
and unstructured data.
But when you combine it
with Azure Functions,
Azure Cosmos DB makes
storing data quick and easy,
with much less code than storing
data in a relational database.
Matthias is here to show some of
the best practices
today on Azure Friday.
[MUSIC].
>> Hey folks, I'm Scott Hanselman,
and it's Azure Friday.
I'm here with Matthias bringing
me some of my favorite things,
Cosmos DB and Azure Functions.
So like chocolate and peanut butter,
only good things can
happen. How are you, sir?
>> Pretty good. Thank you Scott.
So for today's demo,
I want to show you
how you can leverage
Azure Cosmos DB Multi-master,
and change these features to create
serverless sure replicated
even based architectures.
So what I did is, I deploy
two groups of devices.
One in North Europe,
and one in Central US,
and I'm sending that telemetry
into my Cosmos DB Database.
But I want to do it as
quickly as possible.
So what I did is, I created
my Cosmos DB Database.
I deployed it in distributions,
both in North Europe
and in Central US,
and I've enabled
the Multi-master capability.
This make it so.
Both endpoints are both,
read and write enable.
>> Okay, so let me get
my head around this.
You're writing data from
devices all over the world,
like globally writing?
>> Yeah.
>> In my mind,
which is like how we
used to do things,
you can have lots of reads,
but you only get to
write to one place.
>> Exactly.
>> You said you're writing
to multiple places too.
>> At the same time.
That is quite amazing.
So what I also did is,
I took this code that I used
to build an Azure Function,
I'm using the HTTP trigger,
to receive the telemetry
from the devices,
and then I'm using the Cosmos DB
output binding to save it.
Now there are
two important properties
that I'm sitting on
the output binding.
One is the
UseMultipleWriteLocations to two.
This will make it so,
the binding will know
that I want to leverage
the Multi-master capability
in my Cosmos DB Database,
and the other one is
the prefer locations.
Now if you take a closer look,
I'm not using a constant value,
I'm using the placeholder.
That makes it so, I can take
this code and I can deploy
it in as many function apps,
in as many regions that I want,
and if I set an environment variable
that will take the [inaudible].
Where in which reason this
is running, it will make it,
so the output when it will connect to
the local Cosmos DB endpoint
and two local rights.
>> Okay, so the rights
are as close to
one of the Writable
Locations as possible.
>> Exactly, and I can
even use App Insights to
monitor both my Central US,
and my North Europe
function instances,
to see what is the latency in
the operations in both endpoints.
The important thing
with function is that,
since these are
different functions app,
they can scale independently
based on the needs of my devices.
So if I get more throughput
on a particular region,
then that function app can scale into
more instances to
absorb that data flow.
>> Interesting. So if I'm going to do
a global application like this,
depending on what I'm trying to do
whether it's IoT data,
financial data,
or website data, I might
have it done by time zone.
>> Yeah.
>> Europe is waking up,
and then Cosmos will wake
up and scale with Europe-
>> Yeah.
>> -and then they go to sleep,
and then Asia wakes up.
>> Exactly.
>> I can deploy these functions up,
and I can use the same
code in all the regions.
So another thing I wanted to
show you during this demo,
is a Real-time Dashboard.
Because let's face it,
Real-time Dashboards are pretty cool.
So what I did is,
I created this very small
and simple console app,
that it's receiving all real time,
all the writes that are happening
in all Microsoft DB regions.
Now how did I build this?
I went and use the Cosmos DB
trigger in Azure functions.
The Cosmos DB trigger is leveraging
the change feed feature.
The change feed is like a timeline,
where all the events that happened in
the past and now in
your container are stored.
So technically you could hook into
the latest known endpoint
on this timeline,
and start varying up functions
whenever there is something
new in that timeline.
So what I'm doing here is,
I'm varying up my function,
whenever there is new telemetry
data in my container,
and then using
the Singular Output Binding.
As you may know, Signal R uses
WebSockets connections to send
push notifications on real time.
So my war application is just opening
a WebSocket to my singular hub,
and it's receiving all real-time,
all the changes that are happening
in my Cosmos DB Database.
>> Interesting, and also
you've effectively got
a change log that you're
listening a firehose,
that you just listen into,
but your hearing the events all over.
>> Exactly. It doesn't matter
which region is being written to,
I'm listening to all the events.
>> The other thing that
I'm noticing by the way,
which I think it's just interesting
because of the way that
you're choosing to architect
your application using Cosmos DB
a Serverless Global Database
in Azure functions.
A Serverless Compute Mechanism.
You've really shown
me very little code.
You're using input
and output bindings
in a very clever way that's allowing
you in multiple places to
lose a dozen lines of code,
and not interesting code just boring.
Read this, write that.
The output bindings for
Cosmos DB is very clean.
>> Yeah I can focus on what I
really want to do with my function,
and all that orchestration
that is really happening in debug like
opening the connection, [inaudible].
The connection is
entirely done for me.
>> That's cool.
>> Another thing I can also do,
is I can also use
Application Insights to monitor
this trigger and see which are
the course remaking both
to the cognitively instance
onto signaler,
and I can see that I'm only
reading from one Cosmos DB region.
But I'm getting all the data that
is coming from both regions.
>> You're using the Azure
SignalR Services as well,
which is again another serverless.
>> Exactly. So basically I have
no applications running
in terms of the user-
>> Additionally.
>> Exactly.
>> Yeah.
>> I have all these serverless
functions running constantly.
Finally one thing that I
wanted to show you is,
it's really common for
our customers to have,
when they want to
analyze the information,
they have different query needs,
so that the queries are
different from the queries,
and the operations you do
on the record data path.
So what I did is,
I have this materialized
view container,
and my material is
view documents or the schema,
is different from the schema that I'm
using to store
the telemetry information.
Maybe I want to do like
an aggregation over the data,
and my partition key on that
particular container is different.
So how do I make
this synchronization between
two containers online
and on real time.
What I use to do this,
I have these very simple,
another Cosmos DB trigger
that is working in parallel,
and totally independent from
the previous trigger
for the notifications,
and I'm mixing it with
the Cosmos DB output
binding targeting
the separate container.
So basically what this function is
simply doing is just
taking all the telemetry,
then group with the information
candidate in my aggregation,
and then sending that new scheme
of information to
my second container.
So basically I'm maintaining
both containers in sync on
real-time and with probably
like five lines of code.
>> So is the reason for doing that.
I mean, I'm thinking simplistically
you're duplicating a lot of data,
but at the same time you're
also optimizing it that,
the data ingress for the multiple
writes in multiple locations,
is optimized and then,
the heavy query, the heavy reads,
the complicated reads are in
a totally different collection which
allows Cosmos to be smarter
and not fight with itself.
>> Another reason is, maybe
the queries that I want to do on
my materialized views have
a different partition distribution.
So I want my queries to
be as fast as possible,
regardless of how I'm
storing the hot data.
So I can separate both of my units.
>> Then you could ultimately
build a data pipeline,
and move that data
farther and farther,
into more appropriate places
and data breakers
your data factories.
In keeping everything
partitioned away,
and even put it into
a Dashboard another dashboard.
>> Another dashboard.
>> Power BI dashboard
which is also serverless.
>> This combo fusing the trigger
and the output binding,
is also useful when
people create a container that
you find a partition key,
and then after a while they realize
that they picked the wrong partition
key. It's really common so-
>> Yeah, that is very common.
>> Exactly, and one thing
they can do is,
they can create a function
using the trigger.
Said these particular flag call
start from beginning to true,
into a trigger and mix it
with this output binding to
a separate collection with
a new and correct partition key,
and they start from beginning
through a flag will make it so.
The trigger instead of starting
from now to listen for changes,
it will go back in time and replay
all the inserts that happen
on that container ever.
>> Wow.
>> So it's a hot online
migration of your data,
from one container to another,
and you can do this
without any downtime.
>> [inaudible] in my mind
while you were saying that,
I was imagining all of
these cups of liquid,
and how there you're moving
from one cup to another cup,
but you're not dropping
any liquid at all.
>> Exactly, and I can do this on
real time without any downtime
on my containers.
>> Fantastic. So we can
learn more about this where,
and we can play with Cosmos on.
I know we can do Cosmos for
free, right? Try Cosmos.
>> Yeah. There is a try Cosmos
offering that lets you play with
Cosmos DB for 30 days without
any credit card or
actual subscription.
You just need a Microsoft account.
>> Do you have
this particular application
in GitHub, or can you
put it there for me?
>> Yeah sure thing, I can do that.
>> Oh, fantastic. All right.
I am learning all about
best practices with
Azure Cosmos DB and Azure Functions,
today on Azure Friday.
[MUSIC]
