WELCOME, EVERYONE. TODAY IS DAY 
THREE OF BUILD, HOPEFULLY YOU GUYS 
DIDN'T PARTY TOO HARD LAST NIGHT. 
WE'RE GOING TO BE TALKING ABOUT 
SOMETHING WE'RE VERY PASSIONATE 
ABOUT, MY NAME IS ANDREW LIU, I'M 
A PM ON THE COSMOS DB DB TEAM. >> 
I'M MARK BROWN, A PM ALSO. >> AND 
I'M TERRY DUCHASTEL, FROM CITR I 
X SYSTEMS. >> TODAY WE'RE GOING 
TO TALK ABOUT A PRETTY COMMON SCENARIO 
FOR US, MULTI-TENANT APPS, AND WHEN 
YOU LOOK AT MULTI-TENANT APPLICATIONS, 
TYPICALLY THERE'S GOING TO BE THREE 
HIGH LEVEL CORE CONCEPTS THAT YOU 
REALLY WANT TO GET RIGHT. THE THREE 
CONCEPTS ARE GOING TO BE HOW DO 
YOU HANDLE PERFORMANCE ISOLATION 
AND MANAGE THAT WITH RESPECT TO 
COSTS; THE SECOND IS GOING TO BE 
IF YOU HAVE A GLOBAL USER BASE, 
AS WELL AS HUGH STRAIGHT, WHEN -- 
HOW DOYOU MANAGE THE TRADEOFFS BETWEEN 
CONSISTENCY, AVAILABILITY AND LATE 
ENSUN, AND THE THIRD ASPECT IS GOING 
TO BE AROUND SECURITY, HOW DO YOU 
MAKE SURE THAT YOU CAN MANAGE THINGS 
LIKE LEAST PRIVILEGED PRINCIPLES 
SO YOU CAN HAVE GOOD SECURITY ISOLATION 
AND WEAR GOING TO TALK ABOUT THIS 
IN THE CONTEXT OF A TOKEN BROKER 
WHICH WE'VE DEPLOYED LIVE AND TERRY 
DUCHASTEL ARE GIVE YOU AN AWESOME 
VIEW ON HOW CITRIX HANDLES THIS 
WITH COSMOS DB. I ASSUME EVERYONE 
HERE HAS FAPT WITH COSMOS DB ALREADY. 
IF NOT, I'M GOING TO QUICKLY LEVEL 
SET. I'M NOT GOING TO SPEND MORE 
THAN A MINUTE ON THIS. THE SHORT 
JUST HERE, THIS IS A DISTRIBUTED 
DATABASE. WE FOCUS ON A LOT OF CORE 
TENANTS AROUND PARTITIONING, REPAIR 
ANY INDICATION AND RESOURCE GOVERNOR 
DEMANDS, AND MAY TEAM LIKES TO CALL 
THIS BEING A HOUSE DAY GRAM. THE 
PARTITIONING, REPLICATION AND RESOURCE 
GOVERNANCE GIVES US A SET OF CORE 
CAPABILITIES, ELASTIC STORAGE AND 
THROUGHPUT, GUARANTEED LOW YOU LATENCY, 
CONSISTENCY MODEL, THIS IS OUR BACKEND 
PLATFORM. WE CAN EXPRESS VARIOUS 
DIFFERENT DATA MODELS ON TOP OF 
THIS BACKEND, INCLUDING THE KEY 
VALUE, COLUMN FAMILY DOCUMENT AND 
GRAPH DATA MODELS, AND THESE GET 
-- THESE ARE MADE AVAILABLE THROUGH 
A VARIETY OF DIFFERENT APIs, INCLUDING 
OUR OWN CORE SQL API, AS WELL AS 
A BUNCH OF OPEN SOURCE BASED API, 
INCLUDING TABLES, CASSANDRA, MONGO 
TB, AND GREMLIN, AND THEN WITH OUR 
RECENTLY NEWLY ANNOUNCED API FOR 
KUBERNETES CLUSTERS AS WELL AS A 
SPARK API FOR DOING HYBRID TRANSACTIONAL 
ANALYTICAL PROCESSING. NOW, LET'S 
JUMP INTO THE FIRST CONCEPT. PERFORMANCE 
PERFORM PERFORMANCE ISOLATION,. 
WE MANAGE THIS IN A COST EFFECTIVE 
MANNER SO THAT WE DON'T HAVE NOISY 
NEIGHBOR PROBLEMS. NOW, TO BEGIN 
WITH THIS, IF YOU HAVEN'T LOOKED 
AT NOISY NEIGHBOR PROBLEMS BEFORE, 
I'M GOING TO BEGIN WITH A SIMPLE 
ANALOGY, AND HAS ANYONE HERE HAD 
A ROOMMATE BEFORE? YEAH, LIKE MOST 
OF US HAVE HAD HOUSES WITH ROOMMATES 
BEFORE. SO TO MAKE AN ANALOGY, I 
HAVE A HOUSE HERE, IT'S A MILLION 
DOLLARS HOUSE, IF I HAVE A SINGLE 
TENANT WITHIN THAT HOUSE, WELL, 
I'M GOING TO HAVE TO TELL THIS TENANT, 
YOU KNOW, WHAT, THE COST COSTS A 
MR. DOLLARS, ONLY ONE TENANT TO 
SPREAD IT AROUND, A MILLION DOLLARS 
FOR THAT TENANT. OF COURSE, THE 
INTUITIVE THING IS, I CAN INCREASE 
THE NUMBER OF TENANTS TO SPREAD 
THAT COST ACROSS TENANTS, AND SO 
IF I WERE TO DOUBLE THE IN YOUR 
OPINION OF TENANTS, I CAN DECREASE 
THE COST PER TENANT BY HALF. AND 
THIS IS GOING TO BE A REALLY FUN 
ROWDY HOUSE, WE'RE GOING TO PACK 
A THOUSAND TENANTS IN HERE, AND 
THAT WAY WE CAN REALLY FLOOR THE 
COST PER TENANT, BUT THE CHALLENGE 
HERE IS THE NOISY NEIGHBOR PROBLEM. 
WHAT IF CAROL REALLY LOVES, I DON'T 
KNOW, JUST POOR MUSIC TASTE, WE 
HAVE THIN WALLS IN THE HOUSE, AND 
SHE'S BLARING MUSIC AND ALLISON, 
BOB AND THE FINE HUNDRED OTHER TENANTS, 
NOT SO HAPPY. SO, HOW DO WE THINK 
ABOUT ISOLATING THESE? ONE MODEL 
OF ISOLATING THESE IS TO PUT THEM 
EACH IN A DIFFERENT HOUSE, IF I 
HAVE A THOUSAND TENANTS, BUT THEM 
IN A THOUSAND HOUSES, BUT I'M BACK 
AT THE PROBLEM OF, WOW, IT REALLY 
GOT COSTLY PER TENANT. SO TODAY 
WHAT WE'RE GOING TO TALK A LITTLE 
BIT ABOUT IS THE VARIOUS KNOBS AND 
LEVERS YOU HAVE TO MANAGE THIS PERFORMANCE 
ISOLATION STILL IN A COST EFFECTIVE 
MANNER. IT TURNS OUT THAT WHEN YOU'RE 
LOOKING AT SERVERS, NOT TOO DIFFERENT 
FROM HOUSING. NOW THAT WE'RE ALL 
READY TO BECOME LANDLORDS -- WHO'S 
READY TO BECOME A LANDLORD? I'M 
SURPRISED PEOPLE RAISED THEIR HAND. 
LET'S GO INTO REAL ESTATE. WE'RE 
TALKING ABOUT SERVERS, WHEN YOU 
BUILD MULTI-TENANT SERVERS, SAME 
THING, IF YOU HAVE, LET'S SAY, A 
THOUSAND TENANTS PACKED ON THE SAME 
BOX, ONE TENANT RUNS AN EXTREMELY 
EXPENSIVE WORK LOAD, USES A LOT 
OF C'S U, A LOT OF MEMORY, HOW DO 
YOU MAKE SURE THAT DOESN'T DEGRADE 
THE PERFORMANCE FOR THE OTHER TENANTS? 
AND IF YOU BREAK IT DOWN TO A SERVER 
PER TENANT, YOU'RE BACK AT THIS, 
THIS GOT REALLY COSTLY REALLY FAST. 
SO, SOME BACKGROUND CONTEXT REGARDING 
COSMOS DB. HOW DO WE APPLY THIS 
TO COSMOS DB? FOR THE FOLKS WHO 
HAVEN'T LOOKED AT COSMOS DB BEFORE, 
VERY QUICK LEVEL SET AGAIN, WHEN 
YOU PROVISION A COSMOS DB ACCOUNT, 
WE HAVE THIS RESOURCE MODEL THAT 
WE START WITH A DATABASE ACCOUNT, 
THAT'S THE INSTANCE THAT YOU CONNECT 
WITH OF THE UNDERNEATH THAT WE CAN 
PROVISION A SET OF DATABASES AND 
UNDERNEATH THAT A SET OF CONTAINERS. 
EXPLAINING CONTAINERS REAL QUICK 
FOR THOSE WHO HAVE MADE WITH COSMOS 
DB, A LOT OF TIMES YOU'LL SEE THE 
WORD COLLECTION, YOU'LL SEE THE 
WORD TABLE, YOU'LL SEE THE WORD 
GRAPH. BECAUSE WE SUPPORT DIFFERENT 
DATA MODELS WHEN YOU'RE TALKING 
TO LET'S SAY A DOCUMENT ORIENTED 
API, WE LIKE TO CALL IT A COLLECTION, 
WHEN YOU'RE TALKING TO ONE OF OUR 
GRAPH APIs, WE LIKE TO CALL IT GRAPH. 
I'M USING THE WORD CONTAINER NOT 
LIKE A DOCKER CONTAINER BUT A DATA 
MODEL WAY TO REFERRING TO A THING 
THAT CONTAINS A BUNCH OF DATA. AND 
UNDERNEATH THAT WE ACTUALLY STORE 
OUR ACTUAL DATA, WHETHER THAT IS 
A DOCUMENT, A ROW, A NODE OR AN 
EDGE, AND I TYPICALLY REFER TO THESE 
AS ITEMS. NOW, WHAT COSMOS DB DOES, 
ONCE YOU HAVE THIS CONTAINER, THINK 
OF IT AS A BAG OF ITEMS, YOU'RE 
GOING TO SET UP A PARTITION KEY 
AND UNDERNEATH THE HOOD WHAT WE 
HAVE HERE IS A DISTRIBUTED SYSTEM. 
WHAT WE'RE GOING TO DO IS SET UP 
DIFFERENT PARTITION SETS, WHICH 
ARE COMPOSED OF PHYSICAL PARTITIONS, 
WHICH ARE THEN COP POSED OF REPLICAS. 
WE'RE GOING TO TALK A LITTLE BIT 
ABOUT THE TRADEOFFS BETWEEN CONSISTENCY, 
LATENCY AND STRAIGHT AND A LITTLE 
BIT LATER IN THE TALK, BUT JUST 
HERE YOU WANT TO SET UP ONE OF THESE 
REPLICA SETS PER REGION PER PARTITION 
SET. BUT THEN YOU'RE GOING TO WANT 
TO THINK ABOUT HOW DO I ACTUALLY 
PARTITION MY TENANTS ACROSS THIS 
MODEL. THE OTHER KEY THING TO UNDERSTAND 
ABOUT COSMOS DB IF YOU HAVEN'T LOOKED 
AT THIS BEFORE IS HOW WE MODEL THE 
PHYSICAL RESOURCES FOR DOING COMPUTE, 
WHENEVER A REQUEST COMES IN OF THE 
AND SO IF YOU HAVE MULTIPLE DIMENSIONS 
LIKE CPU, MEMORY AND IO, WE NORMALIZE 
THIS INTO A REQUEST UNIT WHERE WE 
BASELINE A REQUEST UNIT AS THE MACHINE 
COST OF RUNNING A SINGLE POINT READ 
ON A ONE KILOBYTE RECORD AND THEN 
EVERYTHING IS PROPORTIONAL FROM 
THERE BASED OFF OF THE REQUEST. 
NOW, WHEN WE LOOK AT RUs, WHAT YOU'LL 
FIND VERY QUICKLY IS WE HAVE MULTIPLE 
WAYS OF PROVISIONING THESE RUs. 
WE CAN PROVIDE THEM AS A DATABASE 
LEVEL AS WELL AS WE CAN PROVIDE 
THEM AT A COLLECTION LEVEL OR A 
CONTAINER LEVEL. WHEN YOU'RE LOOKING 
AT A DATABASE LEVEL WHAT YOU SAY 
REALLY HAPPENING IS, WE'RE PROVISIONING 
A CLUSTER OF MACHINES AND ALL OF 
THESE DIFFERENT CONTAINERS ARE GOING 
TO BE CO-LOCATED ON THAT SAME SET 
OF PHYSICAL PARTITIONS. THEY'RE 
SHARING THE SAME SET OF RESOURCE. 
THERE'S SOME BENEFITS FOR DOING 
SO, THERE'S ALSO SOME TRADEOFFS 
FOR GETTING THIS BENEFIT. IN CONTRAST, 
WHEN YOU DO CONTAINER LEVEL RU, 
WHAT YOU SEE IS THE MAPPING FROM 
ACCOUNT HER AND ITS LOGICAL PARTITIONS 
TO A PHYSICAL PARTITION, IT'S GOING 
TO BE ISOLATED. WE'RE GOING TO BE 
ABLE TO HAVE INDEPENDENT RU KNOBS 
ON EACH CONTAINER AND THEY'RE GOT 
SCALE INDEPENDENTLY OF ALL OF THE 
OTHER CONTAINERS AS WELL AS NOT 
BE ABLE TO TAP INTO THE AWESOME 
SET SAME SETOF RESOURCES. HERE ARE 
THE OPTIONS YOU CAN LOOK AT WHEN 
BUILDING A MULTI-TENANT APP, RANGING 
FROM LEFT TO RIGHT. WHAT WE HAVE 
HERE IS A FEW OPTIONS WE HAVE ARE 
STORING A TENANT PER DATABASE ACCOUNT 
-- SORRY, A DATABASE ACCOUNT PER 
TENANT, PROVISIONING A CONTAINER 
DEDICATED THROUGH PER TENANT, A 
CONTAINER WITH SHARED THROUGHPUT 
PER TENANT AND A PARTITION KEY PER 
TENANT. AS YOU MOVE FROM LEFT TO 
RIGHT, BASICALLY WHAT YOU WANT TO 
DO IS T SURETY SIZE YOUR TENANTS 
BECAUSE ON THE LEFT SIDE WE HAVE 
INCREASED ISOLATION BUT AT THE SAME 
TIME INCREASED COST. AT THE FAR 
RIGHT SIDE WE'RE GOING TO HAVE LOWER 
APPOINT OF AMOUNT OF ISOLATION, 
LOWER COST PER TENANT, AND DEPENDING 
ON THE SIZE OF TENANTS AND YOUR 
SCENARIO, YOU MIGHT FIND LIKE FROM 
ARE NICE SHADES OF GRAY IN BETWEEN. 
SO I'M GOING TO GO FROM THE LOWEST 
COST OVER TO THE GREATEST COST, 
I'M GOING TO READ FROM RIGHT TO 
LEFT, BUT FROM A PARTITION KEY PE 
TENANT, BASICALLY WHAT YOU'RE DOING, 
YOU'RE COMPLETELY DECOUPLING ALL 
THE PHYSICAL INFRASTRUCTURE TO A 
TENANT. AND THE IDENTIFIES THING 
ABOUT THAT IS THAT FROM A COST PER 
TENANT PERSPECTIVE, THERE IS NO 
COST PER TENANT. IF YOU HAVE STREEPLY 
SMALL TENANTS WHERE YOU NEED TO 
BUILD A COST EFFECTIVE SOLUTION, 
ESPECIALLY IF IT'S A B TO C APPLICATION, 
IF YOU'RE BUILDING -- IF YOU'RE 
BUILDING A SOCIAL MEDIA APPLICATION, 
EVEN A COST OF 1 DOLLAR PER TENANT 
IS PROBABLY NOT PALATABLE. YOUR 
MARGINS WILL BE VERY THIN ON A PER 
TENANT BASIS. IN CONTRAST, IF YOU'RE 
BUILDING SOMETHING LIKE A CLOUD 
SERVICE AND YOU'RE ONBOARDING A 
B TO B INTERPRETS ON TO YOUR PLATFORM, 
THE COST PER TENANT, BECAUSE THEY 
ARE ALSO PAYING YOU A LOT MORE FOR 
GUARANTEES AROUND PERFORMANCE OF 
YOUR SYSTEM, YOU HAVE A HOT MORE 
WIGGLE ROOM IN TERMS OF HOW YOU 
WANT TO ISOLATE THEM. FROM A PARTITION 
KEY PER TENANT PERSPECTIVE, SOME 
OF THE BENEFITS HERE IS BECAUSE 
YOUR TENANTS ARE SHARING THE SAME 
INFRASTRUCTURE, BEYOND COST, THEY'RE 
ALSO SHARING THE THROUGHPUT, AND 
SO IF YOU HAVE SPIKY TENANTS, IN 
WHICH A TENANT MIGHT SPIKE AT ANY 
GIVEN MOMENT, BUT NOT AT ANY SAME 
MOMENT, ALL TENANTS WILL SPIKE, 
THIS IS LIKE AN ANALOGY, LIKE THE 
BANKING PROBLEM. YOU DON'T NEED 
TO HAVE TO HAVE CASH ON HAND FOR 
EVERYONE TO WITHDRAW THEIR MONEY 
BECAUSE YOU WON'T HAVE YOUR ENTIRE 
CUSTOMER PACE BAIL OUT AT ONCE, 
BUT YOU NEED TO HAVE ENOUGH CASH 
ON HAND SO THAT GIVEN A SET OF CUSTOMERS 
COMING IN, YOU NEED TO BE ABLE TO 
HANDLE THEIR TRANSACTIONS. SAME 
THING HERE, WHAT YOU'RE DOING IS, 
IF YOU HAVE, LET'S SAY, A BILLION 
TENANTS, THIS IS A WILDLY POPULAR 
SOCIAL MEDIA APPLICATION, AT ANY 
GIVEN TIME YOU HAVE A MILLION USERS 
SPIKING BUT YOU DON'T REALLY WANT 
TO RECOGNITION FOR A BILLION USERS 
SPIKING, IT IS A PRETTY COST EFFECTIVE 
WAY OF SHARING THAT THROUGHPUT. 
ANOTHER SUED EFFECT SIDE EFFECT, 
BECAUSE THEY'RE SHARING THE SAME 
CONTAINER AND THEY'RE THEY'RE SEGMENTED 
BY PARTITION KEY, BECAUSE CONTAINERS 
ACT AS A BOUNDARY FOR QUERIES, IF 
YOU WANT TO QUERY ACROSS MULTIPLE 
TENANTS OR TRANSACTIONS, DEPENDING 
ON HOW YOU MODEL YOUR PARTITION 
KEY, TO GIVES YOU A MECHANISM FOR 
DOING SO BECAUSE THE CONTAINER DOES 
ACT AS THAT BOUNDARY FOR QUERIES. 
YOU WANT TO THINK ABOUT THE PARTITION 
KEY CAN YOU FEEL BECAUSE -- YOU 
CAN MITIGATE NOISY NEIGHBOR PROBLEMS. 
I'LL TALK ABOUT HOW THESE PARTITION 
KEYS GET PLACED ON THE UNDERLYING 
PHYSICAL INFRASTRUCTURE IN JUST 
A SECOND, BUT AT A HIGH LEVEL, YOU CAN
ALSO DO 
THINGS LIKE GROUPING TENANTS BY A SET OF CONTAINERS 
SUCH THAT, YOU KNOW, IF I HAVE
TENANTS A AND B 
ON CONTAINER ONE AND TENANTS C AND D 
ON CONTAINER TWO, A AND B CANNOT 
TOUCH THE RESOURCES FOR TENANTS 
C AND D. NOW, GOING FROM SMALL TO 
LARGE, IF WE CONTAINERS WITH SHARED 
THROUGHPUT, THIS GIVES YOU A BIT 
MORE EASY MANAGEMENT OF THESE TENANTS. 
YOU CAN CONVENIENTLY DROP A TENANT 
WHENEVER A TENANT LEAVES AS OPPOSED 
TO HAVING DO BULK DELETE OPERATIONS. 
AND THIS ALSO GIVES YOU A NICE WAY 
OF FURTHER MITIGATING THAT NOISY 
NEIGHBOR BLAST RADIUS ON HOW YOU 
GROUP YOUR TENANTS, YOU CAN GROUP 
THEM BY DIAGNOSE AS WELL,. A LOT 
OF MANAGEABILITY ASPECTS. THE BIG 
CHALLENGE IS, AS YOU TRENDING THEN 
THE ISOLATION, THE MINIMUM THROUGHPUT 
FOR A DATABASE DOES SCALE WITH RESPECT 
TO THE NUMBER OF CONTAINERS, WE 
NEED AT LEAST 100 RUs TO POWER EACH 
ONE OF THE CONTAINERS WHEN USING 
THE SHARED THROUGHPUT MODEL, SO 
IT RAISES THE' PER TENANT TO THESE 
$6 PER MONTH ROUGH. THIS IS GREAT 
WHEN YOU'RE DOES MEDIUM-SIZED TENANTS. 
IF YOU'RE DOING A STANDARD OFFER 
FOR B TO B APPLICATIONS. AND IF 
YOU HAVE LET'S SAY A STANDARD SKU 
AND A PREMIUM, FOR THE PREMIUM YOU 
MIGHT WANT DEDICATED. THINK OF A 
VPS VERSUS A VIRTUAL MACHINE. YOU'RE 
GOING TO HAVE TENANTS WHO HAVE -- 
THEY WANT COMPLETE GUARANTEES THAT 
NO ONE ELSE CAN TAP INTO THEIR RESOURCES 
PROVISIONED FOR THEM, THEY WANT 
DEDICATED THINK PUT, RIGHT? ON A 
VPS, ONE TENANT RUNNING AN EXPENSIVE 
WORKLOAD IS GOING TO GO INTERFERE 
WITH EVERY OTHER TENANT ON THAT 
SAME PHYSICAL INFRASTRUCTURE, IN 
CONTRAST, WHEN YOU DO A TED INDICATED 
MODEL LIKE A VM, YOU'RE SETTING 
ASIDE THESE RESOURCES, TAPPED IN 
BY ONE TENANT AND NO OTHER TENANTS. 
WITH CONTAINERS WITH DEDICATED THROUGHPUT, 
YOU'RE NOW GIVING INDEPENDENT THROUGHPUT 
KNOBS FOR EACH CONTAINER SO YOU 
CAN SCALE EACH ONE OF THESE TENANTS 
INDEPENDENTLY. AND THEN YOU CAN 
ALSO GROUP THESE TENANTS WITHIN 
A DATABASE ACCOUNT BASED OFF OF 
REGIONAL NEEDS. THE MAIN DIFFERENCE 
BETWEEN CHAFING CONTAINERS WITH 
DEDICATED THROUGHPUT AND A DATABASE 
ACCOUNT PER TENANT IS A DATABASE 
ACCOUNT IS WHERE WE HOSE ALL OF 
THE GEOREPLICATION KNOBS, AND SO 
IF YOU DO NEED DIFFERENT GEOCONFIGURATIONS, 
YOU CAN DO A DATABASE ACCOUNT PER 
TENANT OR YOU CAN DO, THIS IS A 
DATABASE ACCOUNT FOR MY EUROPEAN 
TENANTS VERSUS THIS IS AN AT ANY 
ACCOUNT FOR LET'S SAY MY AMERICA 
PLUS AMEA TENANTS, JUST FORSAKE 
OF EXAMPLE. BUT AS WE GO AND STRENGTHEN 
THIS, WHAT YOU NOTICE THE COST PER 
TENANT RISES QUITE A BIT MORE, AND 
SO WHAT YOU WANT TO DO IS YOU WANT 
TO COME UP WITH A HYBRID STRONG 
WHERE YOU'RE T-SHIRT SIZING YOUR 
TENANTS, AND A NATURAL DIVISION 
IS, IF YOU'RE DOING A B TO B APPLICATION, 
IF YOU HAVE PREMIUM AND STANDARD 
OFFERS, BASED OF WHETHER IT'S A 
PREMIUM OR STANDARD OFFER, YOU CAN 
GIVE THEM DIFFERENT LEVELS OF ISOLATION. 
ANOTHER WAY TO THINK ABOUT IT IS 
THE NUMBER OF SEATS. SO IF YOU MOSTLY 
-- HAVE AN APPLICATION FOR SMALL 
TO MEDIUM BUSINESSES, A LOT OF MOM-AND-POP 
SHOPS AND SUDDENLY A MEGA-CORP COMES 
IN AND MOM-AND-POP SHOPS HAVE 10 
USERS CONCURRENTLY USING YOUR APPLICATION 
AT ANY GIVEN TIME, WHEREAS MEGA-CORP, 
INSTEAD OF HAVING 10 USERS THEY 
HAVE 100, 000 USERS, THERE'S DIFFERENT 
WAYS OF HANDLING THIS. IF YOU WERE 
TO GO THAT PARTITION KEY PER TENANT 
ROUTE, THERE'S ALSO A DIFFERENT 
TECHNIQUE YOU CAN APPLY, ESPECIALLY 
AROUND SALTING A KEY. OUR SD Ks 
ARE INTELLIGENT ENOUGH TO ROUTE 
A QUERY BASED OFF OF A SET OF PARTITION 
KEY VALUES, IF YOU WERE TO SALT 
LET'S SAY CONTOSO, TO ONE, TWO, 
I CAN ALLOCATE TWICE AS MUCH LOGICAL 
PARTITION BANDWIDTH ON THAT PARTICULAR 
LARGER TENANT. IN TERMS OF RECITES 
REPOSITORY SIZING, RESIZING, MOM-AND-POP 
ARE GOING GANGBUSTERS AND THEY'RE 
LARGER THAN WHAT YOU HAD FORECASTED, 
HOW DO I PERFORM TENANT MIGRATIONS? 
LEVEL OUR CHANGE FEED TO FACILITATE 
THAT. A NICE MECHANISM OF GETTING 
INCREMENTAL READS, SO YOU DON'T 
HAVE TO HAVE DOWN TIME BUT YOU CAN 
DO ONLINE MIGRATIONS, CATCH ANOTHER 
COLLECTION UP WHEN YOU'RE PROMOTING 
THEM TO DEDICATED THROUGHPUT AND 
THEN ONCE THEY'RE CAUGHT UP, YOU 
CAN DO A CUTOVER. NOW, TO GIVE YOU 
KIND OF A VIEW OF WHAT HAPPENS BEHIND 
THE SCENES, HOW WE MAP THESE LOGICAL 
PARTITION KEYS ONTO PHYSICAL PARTITION 
SETS, WE USE A TECHNIQUE CALLED 
CONSISTENT HASHING. IF YOU FIND 
-- WHAT MOST LITERATURE WILL REPRESENT, 
CONSISTENT HASHING IS A RING MODEL. 
BASICALLY ANY TIME YOU HASH A PARTITION 
KEY, YOU'RE GOING TO BE ABLE TO 
MAP THAT TO A RANGE OF VALUES FROM 
END VALUE TO MAX VALUE AND THAT 
WILL WRAP AROUND. THE KEY DIFFERENCE 
BETWEEN CONSISTENT HASHING AND A 
NAIVE HASH IS BASICALLY, WHAT YOU'RE 
DOING IS A RANGE PARTITION ON TOP 
OF THE HASHED VALUES. LET'S SAY 
I HAVE A MODEL WHERE I HAVE PARTITION 
SET A AND B, THAT NEEDS TO GROW. 
WHAT YOU'RE DOING IS ACTUALLY BASED 
OFF OF THE HASH OF THE PARTITION 
KEY VALUE, YOU'RE PLACING THEM SOMEWHERE 
ON THE RING AND THAT MAPS TO EITHER 
A PHYSICAL PARTITION SET A OR B, 
AND FROM A CONSISTENT HASH GO PERSPECTIVE, 
THAT ALLOWS YOU TO SUBDIVIDE PARTITIONS 
THAT ARE HOT SUCH THAT YOU DON'T 
NEED TO DO A COMPLETE REBALANCE, 
RESHUFFLE OF THE WORLD. IMAGINE 
IF I DID A NAIVE HASH, IF I DO A 
HASH MOD TWO AND IT'S GOING TO 12, 
ONE, TWO, IF YOU MAP TO ONE, TWO, 
THREE, THAT'S A BIG EARTHQUAKE, 
YOU WILL SHUFFLE THE WORLD. CONSISTENT 
HASHING GIVES YOU A WAY OF AVOIDING 
THAT SUCH THAT PARTITIONS CAN GO 
AND SCALE OUT INDEPENDENTLY. NOW, 
LOOKING AT HOW THIS MAPS TO OUR 
TALK ON MULTI-TENANCY, IMAGINE THAT 
IF I DID PARTITION KEY PER TENANTS, 
IF I HAVE LET'S SAY ALICE, BOB, 
CAROL AS MY TENANTS, BUT I ALSO 
SALT MEGA-CORP, THIS IS GOING TO 
BE CONTOSO, IT'S A LARGE COMPANY 
AND I BREAK THEM UP INTO PARTITION 
KEYS CONTOSO ONE, TWO, THREE, THAT 
ALLOWS ME TO DO, FOR HOTTER TENANTS, 
I CAN GIVE THEM A BIT MORE THROUGHPUT 
THAN MY SMALL MOM AND MOP SHOP. 
AS YOU STRENGTH THEN THAT TO A DATABASE 
LEVEL OFFER AND SHARE THE THROUGHPUT 
ACROSS CONTAINERS, YOU CAN ALSO 
MAP THAT SUCH THAT WHAT HAPPENS 
BEHIND THE SCENES IS PARTITION KEYS 
BELONGING TO COLLECTION ONE AND 
COLLECTION TWO, ARE GOING TO BE 
CO-LOCATED ON THE SAME PHYSICAL 
INFRASTRUCTURE AS SUCH, AND AS YOU 
STRENGTHEN THAT FURTHER, IF YOU 
WERE TO DO LET'S SAY A CONTAINER 
PER TENANT WITH DEDICATED THROUGHPUT 
OR A DATABASE ACCOUNT PER TENANT, 
REALLY, WHAT YOU'RE DOING HERE IS 
YOU'RE ACTUALLY PROVISIONING DEDICATED 
SETS OF PHYSICAL HARDWARE UNDERNEATH 
FOR WHICH WE CAN SCALE OUT INDEPENDENTLY. 
WITH THAT SAID, THIS IS ONE ASPECT 
OF MULTI-TENANCY, AND THE NEXT ASPECT 
WE WANT TO TALK ABOUT IS HOW TO 
GUARANTEE HIGH AVAILABILITY AND 
MANAGE THE TRADEOFFS ASSOCIATED. 
>> THANKS, ANDREW. I WANT TO TALK 
ABOUT SOME OF THE FUNDAMENTAL CONSENTS 
KEY TO UNDERSTANDING HOW TO BUILD 
DISTRIBUTED APPLICATIONS, SYSTEMS 
AND DATABASES IN THE CLOUD. THREE 
CONCEPTS I'M GOING TO COVER, THOSE 
ARE CONSISTENCY, LATENCY, AND AVAILABILITY. 
AND WHETHER YOU KNOW IT OR NOT, 
EACH OF THESE CONCEPTS IS FUNDAMENTALLY 
TIED TO EACH OTHER, AND THERE ARE 
SOME REAL ESSENTIAL TRADEOFFS AND 
DECISIONS YOU NEED TO MAKE WHEN 
YOU'RE BUILDING APPLICATIONS, THEY'RE 
GOING TO RUN IN A DISTRIBUTED FASHION. 
SO BEFORE I GET STARTED, I WANT 
TO DO SOMETHING, I WANT TO DEFINE 
CONSISTENCY HERE. WHO RECOGNIZES 
CONSISTENCY IN THIS CONTEXT? RIGHT? 
SURE. EVERYBODY. WE ALL GREW UP 
WITH THIS. BUT IN FACT IN DISTRIBUTED 
SYSTEMS, THIS IS ACTUALLY NOT WHAT 
WE MEAN WHEN WE TALK ABOUT CONSISTENCY. 
IN FACT, CONSISTENCY IN A DISTRIBUTED 
SYSTEM OR DATABASE MEANS THE UNIFORMITY 
OF DATA BETWEEN REPLICAS THAT ARE 
SEPARATED BY TENS, HUNDREDS, OR 
EVEN THOUSANDS OF MILES, SO IT'S 
A DIFFERENT CONCEPT IN TERMS OF 
WHAT WE MEAN BY CONSISTENCY. SO 
IN 1998, A COMPUTER SCIENTIST PUBLISHED 
A PAPER THAT BAKE KNOWN AS THE CAP 
THERM, AND IT POSTULATED THAT FOR 
ANY DISTRIBUTED DATA STORE OR SYSTEM, 
YOU CAN ONLY GET TWO OUT OF THE 
FOLLOWING THREE THINGS. CONSISTENCY, 
SO I ALWAYS GET THE MOST RECENT 
WRITE, OR I GET AN ERROR BACK. AVAILABILITY, 
SO EVERY REQUEST I SEND TO THE SYSTEM 
IS GOING TO GIVE ME A RESPONSE, 
EVEN IF IT'S NOT THE LATEST DATA, 
OR PARTITION TOLERANCE, AND THAT 
IS THAT THE SYSTEM CONTINUES TO 
OPERATE DESPITE AN ARBITRARY NUMBER 
OF MESSAGES BEING DROPPED OR DELAYED 
BETWEEN THE NETWORK OR ON THE NETWORK 
BETWEEN THE NODES. SO SIMPLY PUT, 
CAP SAYS THAT IF A NETWORK GOES 
DOWN, I HAVE TWO CHOICES. I HAVE 
TO CHOOSE FOR AVAILABILITY FOR THAT 
SYSTEM OR I HAVE TO CHOOSE FOR CONSISTENCY. 
SO, I'LL GIVE YOU AN EXAMPLE HERE. 
I WILL TO ILLUSTRATE THIS. IN THIS 
EXAMPLE HERE, I'M GOING TO CHOOSE 
AVAILABILITY FOR MY SYSTEM. SO WHAT 
HAPPENS IS, I WRITE SOME DATA ONTO 
ONE OF THE NODES AND THEN IT REPLICATES 
TO THE OTHER NODE THERE. SO WHEN 
THE NETWORK IS WORKING FINE, THIS 
IS GREAT, RIGHT? THIS IS WHAT WE 
WOULD EXPECT TO HAPPEN. BUT WHAT 
HAPPENS IF THE NETWORK IS CUT IN 
A SCENARIO WHERE I WANT TO CHOOSE 
AVAILABILITY OVER CONSISTENCY BETWEEN 
THESE DIFFERENT NODES? WELL, IF 
I WRITE DATA TO ONE OF THE NODES 
IN THERE, AND I READ IT FROM ANOTHER 
NODE, I HAVE TO GIVE UP CONSISTENCY, 
I CAN'T PHYSICALLY REPLICATE THE 
DATA BECAUSE THEY'RE NO LONGER CONNECTED, 
I'VE GOT A PARTITION BETWEEN THEM. 
LET'S CHOOSE CONSISTENCY IN THIS 
MODEL. IN THIS MODEL HERE, WHAT 
HAPPENS IS, I WRITE DATA, AND THEN 
IT REPLICATES. THAT'S FINE, RIGHT? 
WE'RE CONSISTENT, WE'RE ALSO AVAILABLE, 
BUT WE DON'T HAVE PARTITION BETWEEN 
THE NODES. IF I CUT THE NETWORK 
BETWEEN THOSE TWO NODES NOW, AND 
WRITE SOME MORE DATA, WHAT HAPPENS? 
I GET AN ERROR. I CAN'T REPLICATE 
THE DATA. I CAN'T REMAIN CONSISTENT 
BETWEEN THOSE NODES AND THE APPLICATION 
SO I HAVE TO TRADE ONE FOR THE OTHER 
HERE. I HAVE CHOSEN CONSISTENCY, 
I MUST MAINTAIN CONSISTENCY BETWEEN 
ALL THE NODES WITHIN THAT DISTRIBUTED 
APPLICATION OR DATA STORE, SO IF 
THE NETWORK IS CUT, AND I HAVE A 
PARTITION, THEN I BASICALLY LOSE 
AVAILABILITY. SO, CAP THERM IS GREAT 
IN DESCRIBING WHAT HAPPENS WHEN 
THERE'S A PARTITION BETWEEN THE 
NODES, BUT IT DOESN'T DESCRIBE EVERYTHING. 
SO IN 2010, THERE WAS AN ADDITION 
OR AN ADDENDUM TO THE CAP THERM 
CALLED PACK LC, AND IT SAYS ESSENTIALLY 
IN A PARTITION, YOU HAVE TO CHOOSE 
BETWEEN AVAILABILITY AND CONSISTENCY; 
HOWEVER, EVEN WHEN THE NETWORK STILL 
WORKS, YOU STILL HAVE TO MAKE A 
CHOICE BETWEEN LATENCY AND CONSISTENCY. 
SO THAT'S WHAT THIS LOOKS LIKE IF 
YOU HAVE A DECISION TREE HERE, RIGHT? 
WHEN I HAVE A PARTITION, CAP THERM 
SAYS CHOOSE BETWEEN AVAILABILITY, 
GET SOME ANSWER, IT MAY NOT BE CONSISTENT 
OR I CHOOSE CONSISTENCY, I'M ALWAYS 
CONSISTENCY BUT I MAY NOT BE AVAILABLE. 
WHEN I DON'T HAVE A PARTITION, SO 
EVERYTHING IS WORKING FINE, I STILL 
HAVE TO CHOOSE BETWEEN THAT LATENCY 
FOR THAT APPLICATION AND CONSISTENCY. 
SO LET ME ILLUSTRATE THIS. SO IN 
THIS MODEL HERE I'M GOING TO CHOOSE 
CONSISTENCY BETWEEN MY TWO NODES. 
SO WRITE SOME DATA, FINE. BUT IF 
I'M CHOOSING CONSISTENCY HERE, I 
HAVE TO WAIT FOR THAT DATA TO ACT 
AND THEN COME BACK, RIGHT? I CAN'T 
DO ANY OTHER OPERATIONS. I'M LOSING 
SOME LATENCY AND -- INCURRING SOME 
LATENCY THERE WHILE THE DATA IS 
BEING COMMITTED AND REPLICATED TO 
ALL THE NODES IN THE SYSTEM. SO 
HERE WE GO. I'LL SHOW THAT AGAIN. 
LET'S CHOOSE LATENCY. IN THIS MODEL 
HERE I'M OPTIMIZING FOR LATENCY. 
I DON'T CARE IF I'M CONSISTENT BETWEEN 
THE NODES. SO IF I WRITE DATA, FINE, 
AND THEN READ IT, I MAY OR MAY NOT 
SEE THE SAME ANSWER IN ALL THE DIFFERENT 
NODES THERE, RIGHT? THE ADVANTAGE 
THERE IS, I CAN BLAST AS MUCH DATA 
AS I WANT INTO ANY OF THE NODES 
BUT THE RESULT MAY NOT BE CONSISTENT 
ACROSS THE ENTIRE SYSTEM. SO LET 
ME GO A LITTLE MORE INTO READ LATENCY. 
THERE'S TWO TYPES OF LATENCY WE 
WANT TO TALK ABOUT. THERE'S THE 
READ LATENCY FOR THE SYSTEM AND 
THERE'S WRITE LATENCY. LET ME GO 
INTO THE -- TO EXPAND ON LATENCY, 
ANOTHER WAY TO THINK ABOUT IT IS 
PERFORMANCE. THE LOWER THE LATENCY 
THE FASTER THE REQUEST FROM YOUR 
CLIENT AND APPLICATION TO THE DATABASE 
AND BACK ARE GOING TO PERFORM. WITHIN 
A DISTRIBUTED SYSTEM, THE WAY YOU 
DO THIS IS THROUGH REPLICATION, 
THE CONCEPT OF DATA LOCALITY. GET 
THE DATA CLOSER TO THE USERS. IN 
COSMOS, DATA REPLICATED TO YOUR 
REGION IS GOING TO GET LESS THAN 
10 MILLISECOND RESPONSE TEAMS. IF 
YOU'VE USED COSMOS YOU CAN SEE HOW 
EASY IT IS TO ADD ADDITIONAL REGIONS 
WITHIN YOUR APPLICATION. SO I'M 
GOING TO SOME DEMOS FOR YOU GUYS 
BECAUSE I WANT TO DEMONSTRATE THIS, 
WHAT THIS ACTUALLY LOOKS LIKE. YOU 
GUYS SEE THAT OKAY? OKAY. ACTUALLY, 
BEFORE I DO THAT, LET ME SET THIS 
DEMO UP FOR YOU. HERE'S WHAT I'M 
GOING TO SHOW YOU. I'VE GOT A COUPLE 
OF TESTS HERE OR A COUPLE OF SCENARIOS. 
IN THIS ONE I'VE GOT A COMPANY, 
AND THEY HAVE AN APPLICATION IN 
A DATABASE LOCATED IN EAST U. S. 
AND THEY'VE DECIDE THEY WANT TO 
ADD A FRONT OVER IN WEST U. S. TO 
IMPROVE THE LATENCY FOR THEIR CUSTOMERS 
ON THE OTHER SIDE OF THE COUNTRY. 
ANOTHER SCENARIO, THEY'VE DECIDED, 
WE WANT TO ACTUALLY REPLICATE THAT 
DATABASE AND PUT IT BEHIND THE FRONT 
END WE'VE GOT IN WEST U. S. SO THEY'VE 
DECIDED TO DEPLOY A REPLICA OVER 
THERE. I'LL SHOW YOU WHAT THIS LOOKS 
LIKE. HERE WHAT I'M GOING TO DO, 
I'M GOING TO TEST A ONE READS FOR 
MY FRONT END IN THE WEST U. S. AND 
HAYES THAT IT'S PRETTY SLOW. EVERY 
REQUEST TO GET DATA HAS TO TRAVEL 
ALL THE WAY ACROSS THE U. S. TO 
THE DATABASE AND THEN COME BACK 
AGAIN. SO I'M GETTING ON AVERAGE 
ABOUT 79 MILLISECONDS AT THE 99th 
PERCENTILE ON THAT. WHO WANTS TO 
GET WHAT THIS LATENCY IS GOING TO 
BE? WILL IT BE FAST, SLOW, WORSE? 
SUPER FAST. RIGHT? SO NOW YOU'RE 
GOING TO GET THAT LOW LATENCY BECAUSE 
THE DATA IS RIGHT UP BEHIND THERE. 
THE DATA IT HAD TO TRAVEL TO GET 
TO THAT APPLICATION IS MUCH LESS, 
OR TO THE END CLIENT. SO YOU CAN 
SEE THE AVERAGE LATENCY BETWEEN 
THOSE TWO, YOU KNOW, ALL THE WAY 
ACROSS THE U. S. , ABOUT 79, 80 
SECONDS OF AVERAGE LATENCY, VERSUS 
FOUR MILLISECONDS, SO YOUR APPLICATION, 
HOW IS IT GOING TO FEEL OR HOW WILL 
IT RESPOND? IT WILL SEEM FASTER 
BECAUSE THAT LATENCY IS GOING TO 
BE SO MUCH LOWER. WRITE LATENCY. 
IN THE PRECEDING SCENARIO, REPLICATING 
DATA TO THE SECOND REGION HELPS 
WITH THE LATENCY. FOR THE READS 
ON THIS. BUT APPLICATIONS DON'T 
JUST READ DATA, THEY WRITE DATA, 
TOO. SO HOW DO YOU DEAL WITH THE 
WRITE LATENCY? WELL, IN SEPTEMBER 
LAST YEAR AT IGNITE, WE LAUNCHED 
MULTI-MASTER, AND WHAT MULTI-MASTER 
DOES, IT MAKES EVERY REPAIR REPLY 
CALL FULLY WRITABLE. SO EVERYWHERE 
YOU DEPLOY YOUR REPLICA, YOU CAN 
USE THAT AS A MASTER OR WRITABLE 
MASTER AND THAT DATA WILL REPLICATE 
TO EVERY OTHER NODE WITHIN THE SYSTEM. 
OF COURSE, IN ADDITION TO THE READ 
LATENCY YOU'LL GET, WHICH IS GOING 
TO BE LESS THAN 10 MILLISECONDS, 
YOU'RE NOW GOING TO GET LESS THAN 
10 MILLISECONDS WRITE LATENCY AS 
WELL, ALSO COVERED BY A SLA. YOU 
ALSO GET FIVE NINEs AVAILABILITY 
ON THAT REPLICA AS WELL. LET'S SHOW 
YOU AN EXAMPLE OF THIS. OKAY. SO 
HERE I'M GOING TO DO 100 READS AGAIN 
FROM A SINGLE MASTER IN WEST U. 
S. FROM WEST U. S. AS I SHOWED BEFORE, 
SUPER FAST, MY READ LATENCY IS GOOD 
BECAUSE I'VE REPLICATED TO WHERE 
THAT FRONT END IS GOING TO ASK FOR 
DATA. LET'S DO 100 WRIST AGAINST 
THAT SINGLE MASTER DATABASE THAT'S 
GOING TO GO FROM WEST U. S. TO EAST 
U. S. AND AS YOU'D EXPECT, THAT 
LATENCY IS TERRIBLE. 75, 76, THERE'S 
ONE THAT BUMPED UP OVER A HUNDRED. 
SO ON AVERAGE, 76 MILLISECONDS FOR 
THAT LATENCY. NOW, WE'LL TEST 100 
READS FROM A MULTI-MASTER ACCOUNT 
AND OF COURSE THAT'S GOING TO BE 
FAST, I'M REPLICATED IN THAT REGION 
SO IT'S AN IN-REGION REQUEST. LET'S 
TEST 100 WRITES. SUPER FAST, . SO 
NOW I'VE SOLVED THE PROBLEM OF THE 
WRITE LATENCY. I'LL GET GOOD PERFORMANCE 
NOT JUST ON THE READS BY ANY WRITE 
THAT HAPPENS IN EVERY REGION I'M 
REPLICATED INTO, IT'S GOING TO BE 
INCREDIBLY FAST. HERE YOU CAN SEE 
A SUMMARY OF ALL THE LATENCIES IN 
THERE, YOU KNOW, WITHIN SLA, WHEN 
YOU'RE USING MULTI-MASTER FOR WRITES, 
IN REGION, VERSUS LIKE 70 OR EVEN 
GREATER MILLISECONDS IF YOU'RE GOING 
BEYOND THAT. WHOOPS. MY SETUP SLIDES. 
NEXT I WANT TO TALK ABOUT CONSISTENCY. 
THIS IS THE SECOND CONCEPT I WANT 
TO COVER HERE. IN A DISTRIBUTED 
DATABASE CATEGORY, OUR COMPETITORS 
GENERALLY OFFER STRONG CONSISTENCY 
OR EVENTUAL CONSISTENCY OR MAYBE 
STRONG AND BOUNDED STALENESS. COSMOS, 
YOU HAVE FIVE CONSISTENCY LEVELS 
TO CHOOSE FROM. WE HAVE STRONG, 
WHICH IS BASICALLY LINEARIZED WRITES, 
BOUNDED STALENESS, MEANING THAT 
YOUR READS WILL LEAD BEHIND THE 
WRITE -- WILL LAG BEHIND. SESSION, 
LEVEL CONSISTENCY, THIS PROVIDES 
MONOTONIC READS AND WRITES. CONSIST 
PREFIX. YOU'LL GET UPDATES AT A 
RELAXED LEVEL OF CONSISTENCY BUT 
ALL THE UPDATES YOU GET WILL BE 
IN ORDER. AND EVENTUALLY CONSISTENCY 
WHICH AS THE NAME APPLIES, YOU WILL 
EVENTUALLY BE CONSISTENT BUT THERE'S 
NO GUARANTEE OF THE ORDER IN WHICH 
THOSE UPDATES WILL HAPPEN SO YOU 
CAN WRITE FIVE, SIX, SEVEN AND THEN 
ON A REPLICA YOU MAY SEE FIVE, SEVEN, 
SIX. SO LOTS OF CHOICE IN TERMS 
OF THE CONSISTENCY LEVELS YOU CAN 
SET AND THESE ARE SET AT THE ACCOUNT 
LEVEL BY DEFAULT. HOWEVER, WITHIN 
YOUR APPLICATION, YOU CAN ACTUALLY 
GO AND RELAX THAT LEVEL OF CONSISTENCY 
EITHER AT A SESSION LEVEL, SO WHEN 
YOU INSTANTIATE YOUR SDK CLIENT, 
OR EVEN AT THE REQUEST LEVEL, SO 
LET'S JUST SAY YOU WANT TO BLAST 
AT A WHOLE BUNCH OF DATA, YOU DON'T 
REALLY NEED IT TO BE STRONGLY CONSISTENT 
OR ANY STRONG LEVEL OF CONSISTENCY, 
BUT YOU WANT TO IMPROVE THE PERFORMANCE 
OF THAT, LIKE WE SHOWED EARLIER 
WITH PACK LC, RELAXED CONSISTENCY, 
REDUCED LATENCY, SO YOU CAN REDUCE 
THAT CONSISTENCY LEVEL, GET MUCH 
BETTER PERFORMANCE, AND THEN JUST 
DROP THAT CONNECTION OR DROP YOUR 
-- OR LET IT GO BACK UP TO THE DEFAULT 
THERE. ALSO, ANOTHER THING I WANT 
TO POINT OUT IS, FOR ALL THE CONSISTENCY 
LEVELS, THE READS ARE GUARANTEED 
AT 10 MILLISECONDS AND FIVE NINEs 
AVAILABILITY, AN IMPORTANT POINT 
TO KEEP IN MIND. ANOTHER THING I 
WANT TO POINT OUT, WITH RESPECT 
TO LATENCY VERSUS CONSISTENCY, THERE'S 
ACTUALLY A DIRECT RELATIONSHIP BETWEEN 
THESE THINGS, AND IN FACT THE IMPACT 
OF THAT IS MORE PRONOUNCED WITH 
GREATER DISTANCE. THERE'S NOT MANY 
THINGS IN COMPUTER SCIENCE THAT 
TOUCH ON THINGS LIKE THE SPEED OF 
LIGHT, BUT WHEN YOU'RE DEALING WITH 
A DISTRIBUTED APPLICATION THAT'S 
HAVING TO REPLICA AND COMMUNICATE 
OVER HUNDREDS OR THOUSANDS OF MILES, 
THE SPEED OF LIGHT ACTUALLY BECOMES 
A FACTOR WITH RESPECT TO THE LATENCY 
WITHIN YOUR APPLICATION. PACKETS 
CAN'T TRAVEL ANY FASTER THAN THE 
SPEED OF LIGHT, AND THEY'RE GOING 
TO TRAVEL A LOT SLOWER IN MANY WANs. 
EVERY PACKET HAS TO GO THROUGH ROUTERS 
AND OPTICAL SWITCHES AND ALL KINDS 
OF OTHER CRAZY EQUIPMENT YOU COULD 
HAVE, OTHER THINGS IMPACTING THE 
PERFORMANCE OF YOUR NETWORK, SO 
EACH OF THESE THINGS WILL IMPACT 
THE PERFORMANCE OF YOUR APPLICATION 
OR THE PERFORMANCE OF THAT LATENCY 
WITHIN AN APPLICATION WHEN YOU'RE 
REPLICATING BETWEEN LOTS OF NODES 
OVER A GREAT AMOUNT OF DISTANCE. 
I WANT TO POINT OUT, RECENTLY WE 
ENABLED CUSTOMERS TO BE ABLE TO 
ENABLE STRONG CONSISTENCY ACROSS 
REGIONS. NOW, WHEN WE TALK ABOUT 
THE LATENCY VERSUS THE CONSISTENCY, 
YOU OBVIOUSLY ARE GOING TO HAVE 
A TRADEOFF THERE WITH THAT LATENCY 
WHEN YOU'RE REPLICATING ACROSS GREAT 
REGIONS. SO ONE THING TO KEEP IN 
MIND IS, THE GREATER THE DISTANCE 
IN THERE, THE GREATER YOUR LATENCY 
IS GOING TO BE. ONE OTHER CONCEPT 
I WANT TO TALK ABOUT, TOO, IS THE 
CONSISTENCY LEVEL YOU CHOOSE ALSO 
HAS AN IMPACT ON YOUR THROUGHPUT. 
SO, IN COSMOS DB, FOR STRONG LEVEL 
CONSISTENCY, YOUR WRITES ARE GOING 
TO BE A GLOBAL MAJORITY, RIGHT? 
SO IF I'M REPLICATING IN MULTIPLE 
REGIONS AND I HAVE STRONG CONSISTENCY 
SET FOR THAT ACCOUNT, I NEED TO 
COMMIT TO EVERY REGION I'M REPLICATING 
TO, AND RETURN IT BACK TO THE CUSTOMERS. 
IF USING A RELAXED LEVEL, I'LL DO 
A LOCAL MINORITY. AND YOU SAW IN 
ANDREW'S SLIDES EARLIER WHEN WE 
TALK ABOUT A REPLICA SET, THERE'S 
FOUR REPLICAS WITHIN IT, YOU'VE 
GOT A LEADER, A COUPLE OF FOLLOWERS 
AND THEN ANOTHER -- THAT'S WRONG, 
THAT SHOULD BE A FORWARDER, NOT 
A LEADER UP THERE ON THE TOP. SO, 
WITH -- FOR ANYTHING BUT STRONG, 
YOU'RE GOING TO WRITE TO TWO OF 
THOSE, A LOCAL MINORITY WITHIN THE 
REPLICA SET. ON READS, FOR STRONG 
AND BOUNDED STALENESS, YOU'RE GOING 
TO READ, WHOOPS -- WHAT HAPPENED 
TO MY -- WELL, ALL RIGHT. I MISSED 
MY ANIMATION HERE. LOCAL MAJORITY, 
YOU'RE GOING TO READ FROM THREE 
OF THOSE WITHIN THERE, RIGHT? IT 
HAS TO BE A MAJORITY OF THE REPLICAS 
WITHIN THE SET. AND THEN FOR RELAXED 
LEVEL OF CONSISTENCY, CONSISTENT 
PREFIX AND EVENTUAL, YOU'RE GOING 
TO READ FROM A SINGLE REPLICA WITHIN 
THERE. ALL RIGHT. SO LET'S SHOW 
YOU SOME DEMOS. LET ME SET THIS 
ONE UP. I'VE GOT ABOUT FIVE DEMOS 
IN THIS ONE AND A BUNCH OF DIFFERENT 
SCENARIOS -- OR TWO DEMOS AND A 
BUNCH OF SCENARIOS. I'M GOING TO 
SHOW YOU CONSISTENCY VERSUS LATENCY. 
I HAVE AN ACCOUNT WITH EVENTUAL 
CONSISTENCY AND I'M REPLICATING 
FROM WEST U. S. TO CENTRAL, ABOUT 
A THOUSAND MILES DIFFERENCE BETWEEN 
THE TWO. NEXT, I'M GOING TO SHOW 
YOU CONSISTENCY VERSUS LATENCY WITH 
AN ACCOUNT USING STRONG CONSISTENCY 
REPLICATING OVER THAT SAME THOUSAND 
MILES. AND THEN THIRD, I'M GOING 
TO SHOW YOU AN ACCOUNT WITH STRONG 
CONSISTENCY BUT NOW I'M REPLICATING 
TWICE THE DISTANCE, ROUGHLY TWICE 
THE DISTANCE IN THERE. AFTER THAT, 
I WANT TO SHOW YOU CONSISTENCY VERSUS 
THROUGHPUT. SO I'LL SHOW YOU ONE 
EXAMPLE OF WHAT YOUR THROUGHPUT 
WILL BE WHEN USING EVENTUAL CONSISTENCY 
AND WHAT IT WILL LOOK LIKE WITH 
STRONG CONSISTENCY. YOU GUYS LIKE 
DEMOS? IS THAT OKAY? ARE THERE TOO 
MANY DEMOS IN THIS TALK? NO, OKAY, 
JUST CHECKING. I'M GOING TO TEST 
100 WRITES IN WEST U. S., EVENTUAL 
CONSISTENCY LEVEL AND MY REPLICA 
IS A THOUSAND MILES AWAY. ANYONE 
WANT TO TAKE A GUESS AS TO WHAT 
MY LATENCY WILL BE? SLOW OR FAST? 
LET'S SEE. BLAZING FAST. ROUGHLY 
SIX MILLISECONDS PER WRITE. OKAY. 
NOW I'M GOING TO TEST 100 WRITES 
IN WEST U. S., STRONG CONSISTENCY 
LEVEL, MY REPLICA IS 1, 000 MILES 
AWAY. WHAT'S GOING TO HAPPEN? GOING 
TO WRITE IT IN WEST U. S., REPLICA 
IT A THOUSAND MILES AWAY, COMMIT 
IT AND ACT BACK. MUCH SLOWER. NOT 
THAT BIG, LESS THAN A K. AVERAGE 
LATENCY, 62 MILLISECONDS. OKAY? 
NOW, THIRD TEST. DO 100 WRITES, 
STRONG CONSISTENCY, MY REPLICA IS 
2, 000 MILES AWAY. WHO THINKS -- 
JUST GIVE ME A GUESS. WHAT DO YOU 
THINK IT'S GOING TO BE? SAME? LESS? 
DOUBLE THE DISTANCE, DOUBLE THE 
LATENCY. RIGHT? TO CALCULATE WHAT 
YOUR LATENCY IS GOING TO BE WHEN 
USING STRONG CONSISTENCY IN A DISTRIBUTED 
SYSTEM, YOU ONLY NEED TO CALCULATE 
THAT LATENCY TO THE REGION THAT'S 
THE FURTHEST AWAY. SO IT'S THE ROUND 
TRIP TIME FROM WHERE YOU WROTE TO 
WHERE YOU REPLICATE, THAT FURTHEST 
REGION, PLUS 10 MILLISECONDS FOR 
THE COMMIT WITHIN THE REGION ITSELF. 
ALL RIGHT. NOW, I'M GOING TO DO 
100 READS, THIS IS TESTING THROUGHPUT. 
100 READS, EVENTUAL CONSISTENCY 
IN AN ACCOUNT. THIS IS GOING TO 
READ FROM A SINGLE REPLICA. MY AVERAGE 
REQUEST UNIT IS ONE, BECAUSE I'M 
READING, ONE COMPUTE OPERATION THAT'S 
HAPPENING, RIGHT? NOW, I'M GOING 
TO TEST 100 READS STRONG CONSISTENCY 
IN WEST U. S. SO THESE ARE LOCAL 
MINORITY READS, RIGHT? I'M GOING 
TO READ FROM TWO REPLICAS, WHICH 
IS ACTUALLY TWO PIECES OF COMPUTE. 
EVERY REPLICA WITHIN A REPLICA SET 
HAS ITS OWN PIECE OF COMPUTE. SO 
I HAVE TWO COMPUTE OPERATIONS, AND 
JUST TO MAKE THIS CLEAR, THE REASON 
WE DO THAT, THAT'S HOW YOU GUARANTEE 
CONSISTENCY, RIGHT? WITHIN THE APPLICATION, 
RIGHT? I'M TAKING A QUORUM. IS IT 
REPLICATING? IS IT COMMITTED? MY 
AVERAGE RU IS TWO BECAUSE I'M DOING 
TWO READ OPERATIONS AGAINST TWO 
REPLICAS. WHAT HAPPENS IF THEY DON'T 
AGREE? WHAT WE'RE GOING TO DO WITHIN 
COSMOS IS LOOK AT THE LSN FOR THE 
WRITE, AND WHICHEVER ONE IS HIGHER, 
IS THE WINNER WITHIN THERE, BECAUSE 
IT WAS THE MOST RECENT WRITE, THE 
MOST RECENT UPDATE WITHIN YOUR REPLICA 
SET. OKAY? SO THERE'S YOUR TEST 
RESULTS THERE. AVERAGE LATENCY, 
STRONGER THE CONSISTENCY, THE HIGHER 
YOUR LATENCY. HACK LC. THAT'S THE 
-- THOSE ARE THE TRADEOFFS. THIS 
IS WHAT IT REALLY LOOKS LIKE WHEN 
YOU MAKE A DECISION AND CHOOSE WHAT 
DO I CARE ABOUT, DO I CARE ABOUT LATENCY 
OR DO I CARE ABOUT CONSISTENCY? 
LET'S GO BACK TO SLIDES. THIRD CONCEPT, 
AVAILABILITY. SO, ANYBODY THAT'S 
DESIGNED APPLICATIONS FOR AZURE, 
PARTICULARLY ONES THAT RUN IN MULTIPLE 
REGIONS, ARE FAMILIAR WITH SOMETHING 
THAT LOOKS LIKE THIS, RIGHT? AN 
ARCHITECTURAL DIAGRAM. I'VE GOT 
CLIENTS OUT THERE IN THE INTERNET, 
AND IT COULD BE DEVICES, PHONES, 
BROWSERS, WHATEVER, CONNECTING OVER 
THE INTERNET. IF I'M REPLICATED 
OR DEPLOYED INTO MULTIPLE REGIONS, 
I'LL USE LIKE A TRAFFIC MANAGER 
WITHIN THERE, LOAD BALANCING, TO 
REDIRECT MY REQUESTS TO THE CLOSEST 
REGION TO THEM, RIGHT? BECAUSE I 
WANT TO KEEP THAT LATENCY LOW FOR 
THEM. I'M THEN GOING TO HAVE MY 
APPLICATION DEPLOYED INTO MULTIPLE 
REGIONS. I'M GOING TO USE A REVERSE 
PROXY OR A WAF IN FRONT OF THE FRONT 
END OF THE LOCATION TO HANDLE LOAD 
BALANCING AND SSL TERMINATION. WITH 
EACH OF THE LOGICAL TIERS, I NEED 
-- SO I'M GOING TO HAVE RESILIENCY 
IN TERMS OF REGIONAL, BUT WITHIN 
REGION, I NEED RESILIENCY AS WELL. 
WHEN YOU'RE BUILDING AND DEPLOYING 
FOR THE CLOUD, YOU'RE USING COMMODITY 
HARDWARE, INSTANCES CAN FAIL, ISSUES 
CAN HAPPEN, SO I WANT MULTIPLES 
OF THOSE THINGS AS WELL. SO I'M 
GOING TO HAVE MULTIPLE COMPUTE INSTANCES 
IN THERE, I'M GOING TO LOAD BALANCE 
WITHIN THOSE, IF ANY GOES DOWN, 
I'LL TAKE IT OUT OF THE POOL, REPLACE 
IT, SO YOU'RE GOING TO HAVE RESILIENCY 
AT A REGIONAL LEVEL IN CASE OF A 
REGIONAL EVENT AND THEN WITHIN REGION, 
AT AN INDIVIDUAL COMPONENT LEVEL. 
FOR COSMOS, YOU ARE GOING TO HAVE 
BOTH THOSE THINGS. SO OBVIOUSLY 
FOR EACH OF THESE REGIONS YOU'RE 
DEPLOYED IN, YOU WANT A REPLICA 
BACKING UP EACH OF THOSE APPLICATIONS, 
THE FRONTIER AND THE MIDDLE TIER 
WITHIN THERE. WIN REGION AS I DESCRIBEDDIER, 
WE HAVE FOUR COPIES OF YOUR DATA 
RUN WITHIN A PARTITION SET SO WE 
HAVE RESILIENCY. WHEN YOU'RE DEPLOYED 
INTO MULTIPLE REGIONS, YOU'VE NOW 
GOT RESILIENCY AT A REGIONAL LEVEL 
IN CASE OF AT REGIONAL EVENT OCCURRING 
WITHIN THERE. SO WHO HAS SEEN THIS 
CONCEPT BEFORE, RPO/RTO? YEAH? OKAY. 
SO WHEN YOU'RE DESIGNING FOR A DISTRIBUTED 
SYSTEM, YOU KNOW, HIGH AVAILABILITY 
-- HIGH AVAILABILITY IS JUST PART 
OF THE TASK. YOU NEED TO DESIGN 
A BUSINESS CONTINUITY PLAN TO SAY 
WHAT'S GOING TO HAPPEN IN CASE OF 
DISASTER. AND TO DO THIS, YOU NEED 
TO UNDERSTAND THE TOLERANCE FOR 
YOUR SYSTEM AND YOUR BUSINESS IN 
CASE THAT OCCURS. SO WHAT IS YOUR 
TOLERANCE FOR DATA LOSS? HOW MUCH 
DATA CAN I LOSE? OR HOW LONG CAN 
I BE DOWN FOR, RIGHT? SO IN THIS 
SCENARIO HERE, OR THE WAY I'VE ILLUSTRATED 
IT, THIS IS A FUNCTION OF TIME, 
RIGHT? MY RECOVERY POINT OBJECTIVE 
IS EXPRESSED AS A PERIOD OF TIME. 
HOW MANY MINUTES OF DATA CAN I LOSE? 
A SHOULD AN EVENT OCCUR. IN TERMS 
OF RTO ALSO FUNCTIONS AS A VALUE 
OF TIME IS HOW LONG CAN I TOLERATE 
BEING DOWN FOR. SO, THESE ARE OBVIOUSLY 
DRIVEN BY BUSINESS OBJECTIVES, RIGHT, 
IF YOU'RE A RETAIL OUTLET, AND IT'S 
BLACK FRIDAY, WHAT'S YOUR TOLERANCE 
FOR DOWN TIME? PROBABLY CLOSE TO 
ZERO, RIGHT? YOU'RE GOING TO LOSE 
TENS OR HUNDREDS OF THOUSANDS OF 
DOLLARS A MENORAH HOUR. THAT'S YOUR 
WHOLE SEASON THERE. YOUR TOLERANCE 
FOR RTO IS REALLY ZERO, VERSUS RPO. 
HOW MUCH DATA CAN I TOLERATE LOSING 
WITHIN THERE? THESE ARE CRITICAL 
TO UNDERSTAND, WHEN YOU'RE DESIGNING 
A DISTRIBUTED SYSTEM. SO WE PUBLISHED 
-- THIS IS A TABLE WE PUBLISHED 
THAT'S OUT ON OUR DOCS THAT DESCRIBES 
THE RPO/RTO FOR COSMOS DB USING 
VARIOUS MODES, SO SINGLE OR MULTI-MASTER, 
AND THEN ACROSS VARIOUS CONSISTENCY 
LEVELS, AND THEN WHETHER YOU'RE 
DEPLOYED INTO A SINGLE REGION OR 
MULTIPLE REGIONS. SO, FOR A SINGLE 
REGION DEPLOYMENT, YOUR RPO IS GOING 
TO BE ABOUT, WHAT, FOUR HOURS, BUT 
WHAT I REALLY WANT TO CALL OUT IS 
YOUR RTO WILL BE SOMETHING LESS 
THAN A WEEK. IF YOU'RE BUILDING 
A MULTI--- A MISSION CRITICAL APPLICATION, 
WHO WANTS TO DEPLOY IN A SINGLE 
REGION? YOU WOULD NEVER DO SUCH 
A THING. YOU HAVE TO DEPLOY INTO 
MULTIPLE REGIONS TO PROVIDE THAT 
LEVEL OF AVAILABILITY SHOULD A DISASTER 
OR SOME KIND OF EVENT OCCUR THAT'S 
GOING TO IMPACT A REGION. FOR SINGLE 
MASTER MODE, WITHIN MULTIPLE REGIONS 
THERE, USING A RELAXED LEVEL OF 
CONSISTENCY, YOU'LL HAVE AN RPO 
OF LESS THAN 15 MINUTES AND RTO 
LESS THAN 15 MINUTES. IF YOU'RE 
USING AUTOMATIC FAILOVER, THOSE 
-- AND THAT'S USING AUTOMATIC FAILOVER 
THERE. IF YOU'RE USING A MANUAL 
FAILOVER AND YOU DETECT THAT EVEN 
QUICKER, YOU COULD FAILOVER, YOU 
KNOW, AS QUICKLY AS YOU WANT, BUT 
WE GUARANTEE WITHIN 15 MINUTES FROM 
ONCE WE DETECT THAT THE REGION IS 
NO LONGER AVAILABLE. FOR BOUNDED 
STALENESS, AT A SINGLE MASTER MULTIPLE 
REGION, 100 K UPDATES AT ABOUT FIVE 
MINUTES FOR RPO. THE SAME RTO. FOR 
STRONG, OF COURSE, YOUR RPO WILL 
BE ZERO BECAUSE I'VE REPLICATED 
TO ANOTHER REGION, SO IF THAT REGION 
GOES DOWN, I'M SAFE. RIGHT? BUT 
I HAD TO TRADE OFF SOMETHING FOR 
THAT, DIDN'T I? THE LATENCY? AND 
THEN RTO 15 MINUTES. WHERE THINGS 
REALLY GET INTERESTING AND EXCITING 
IS WHEN YOU'RE USING MULTI-MASTER, 
RIGHT? FOR A RELAXED LEVEL OF CONSISTENCY, 
FOR EVERYTHING BUT STRONG, RTO OF 
ZERO. AND WHY IS THAT? WELL, WHAT 
WE'RE DOING IS, WITHIN OUR SDK CLIENT, 
IF WE DETECT A REGION HAS GONE DOWN, 
WE'LL REDIRECT YOUR REQUEST AND 
RETRY IT, IN THE NEXT CLOSEST REGION, 
WHEREVER YOUR APPLICATION OR DATABASE 
IS DEPLOYED. SO RTO OF ZERO WITHIN 
THERE. AND THEN OF COURSE IF YOU'RE 
STRONG, STRONG CONSISTENCY, YOUR 
RPO OF ZERO. YOU MAY WONDER, WHY 
NOT RPO OF ZERO AND RTO OF ZERO? 
ANYBODY WANT TO TAKE A GUESS AS 
TO WHY? COME ON, YELL IT OUT. I 
MAY GIVE YOU A PAIR OF SOCKS LATER. 
>> [INAUDIBLE] >> SORRY? >> [INAUDIBLE] 
>> RIGHT. SO WHAT DOES CAP THERM 
SAY, RIGHT? LET'S GO BACK TO THIS. 
SO, NETWORK GOES DOWN, I HAVE A 
PARTITION. I HAVE TO CHOOSE BETWEEN 
AVAILABILITY AND CONSISTENCY, RIGHT? 
ANOTHER WAY OF LOOKING AT THIS IS, 
RPO EQUALS WHAT? IT'S CONSISTENCY, 
RIGHT? REMEMBER? GO BACK TO -- WHOOPS. 
I LOST MY OTHER THING. I WAS GOING 
TO PUT THAT RPO/RTO THING UP, BUT 
RPO IS CONSISTENCY, RIGHT? WHEN 
I'M CHOOSING STRONG CONSISTENCY, 
RPO 0. RIGHT? AND IN MULTI-MASTER, 
WHEN I'M CHOOSING AVAILABILITY, 
RTO OF ZERO. BUT I CAN'T TO BOTH. 
CAP THERM SAYS THAT'S PHYSICALLY 
IMPOSSIBLE. SO THAT'S IT FOR THIS 
SECTION RIGHT HERE. I WANT TO TRANSITION 
THIS OVER TO TERRY, AND HE'S GOING 
TO TALK ABOUT SECURITY AND A PRETTY 
COOL TOKEN BROKER THAT HE'S DEVELOPED. 
IT'S ALL YOURS. >> CONSTRUCTION 
MARK. SO, I AM FROM CITRIX, I'M 
AN ARCHITECT, AND NOT EVERYONE MAY 
ALREADY BE FAMILIAR WITH CITRIX. 
I WANT TO TAKE A LITTLE BIT OF TIME 
AND QUICKLY GO OVER WHAT WE DO. 
CITRIX IS MOSTLY KNOWN FOR ITS APPS 
AND DESKTOP VIRTUALIZATION. MORE 
RECENTLY WE'RE KNOWN FOR OUR NETWORKING 
AND OUR SIZE DIAL SOLUTIONS. IN 
PARTICULAR, WE HAVE SOMETHING CALLED 
CITRIX CLOUD, A PLATFORM THAT OUR 
CUSTOMERS CAN USE IN ORDER TO MANAGE 
ALL OF THEIR SOLUTIONS. I'M AN ARCHITECT 
WITHIN THE CITRIX END-POINT MANAGEMENT 
GROUP, ONE OF OUR PRODUCTS. IT ALLOWS 
YOU TO MANAGE YOUR DEVICES WITH 
MOBILE DEVICE MANAGEMENT, MDM AND 
SO FORTH. WHAT I REALLY WANTED TO 
TALK ABOUT TODAY IS SOME OF THE 
STUFF THAT WE'RE DOING HERE WITH 
COSMOS DB. AND IN PARTICULAR, REGARDING 
OUR JOURNEY, REGARDING OUR JUST 
ANY TO MULTI-TENANCY. SO WE SAT 
BACK AND WE DECIDED AND ASKED OURSELVES, 
WHAT ARE SOME OF THE MOST IMPORTANT 
THINGS THAT WE WANTED TO CONSIDER, 
RIGHT? THESE WERE SOME OF THE ONES. 
IN PARTICULAR, TENANT ISOLATION 
IS ABSOLUTELY HUGE FOR US. AND YOU 
HEARD FROM ANDREW ON SOME OF THE 
TRADEOFFS. WELL, WE DID MOSTLY -- 
WHAT WE DID WITH OUR STUFF IS THAT 
WE'RE DEVELOPING MORE AND MORE MICROSERVICES, 
SO WE FIND THAT PARTITION KEY TENANCY, 
IN OTHER WORDS, A PARTITION PER 
TENANT, WORKS OUT REALLY WELL FOR 
US. THERE'S A COUPLE BENEFITS, MOSTLY 
BECAUSE OUR MICROSERVICES ARE QUITE 
SMALL, AND THE OTHER ONE IS IT ALLOWS 
US TO DO THE REPORTS AND CROSS TENANT 
QUERIES WHEN WE NEED TO DO. ANOTHER 
THING THAT'S SUPER IMPORTANT IS 
THE CONCEPT OF LEAST PRIVILEGE. 
WHAT YOU DON'T HAVE, YOU CAN'T LOSE. 
AND IN PARTICULAR WHAT I WANT TO 
SHOW IN THE NEXT COUPLE OF SLIDES 
IS A CONCEPT THAT WE CALL THE TOKEN 
BROKER CONCEPT THAT REALLY AIDS 
YOU IN ADDING EXTRA SECURITY AND 
ISOLATION BETWEEN YOUR TENANTS. 
AND FINALLY, HIGH AVAILABILITY. 
SO MARK TALKED A LOT ABOUT, YOU 
KNOW, WHY THAT'S IMPORTANT AND HOW 
TO DO IT. IN FACT, I HAD NEVER MARK 
BEFORE LIKE TWO WEEKS AGO OR SOMETHING 
LIKE THAT, AND THE ARCHITECTURE 
THAT WE HAVE ACTUALLY RESEMBLES 
A LOT OF WHAT MARK SHOWED, WITH 
THE TRAFFIC BROKER AND AN APP GATEWAY 
PER REGION AND SO FORTH. SO THESE 
ARE PATTERNS THAT YOU DO SEE IN 
INDUSTRY. CERTAINLY WHEN YOU THINK 
OF HIGH AVAILABILITY, IT'S NICE 
TO ADD IT AND WHATNOT, BUT THERE'S 
THAT COST, WHEN YOU GO CROSS-REGION. 
WE'RE A GLOBAL COMPANY, AND, YOU 
KNOW, WE SET UP A FRONT END IN NORTHERN 
EUROPE AND WE WERE SEEING 500 MILLISECOND 
ROUND TRIP TIMES. AND THAT IS ABSOLUTELY 
NOTICEABLE TO YOUR END USERS, IN 
SOMETHING LIKE AN ADMIN CONSOLE 
UR OR SOMETHING LIKE THAT. SO WHEN 
THEY RELEASED MULTI-MASTER, WE JUMPED 
ON THAT AND WE HAVEN'T LOOKED BACK 
SINCE. OUR ROUND TRIP TIMES WENT 
DOWN TO 125 MILLISECONDS WITH ALL 
OF OUR ADDITIONAL STUFF, AND THAT'S 
JUST BEEN SUCH AN IMPROVEMENT, AND 
THAT IS ACTUALLY ACCEPTABLE FROM 
-- IN TERMS OF UI RESPONSIVENESS 
AND SO FORTH. SO I'M GOING TO TALK 
NOW ABOUT THE TOKEN BROKER CONCEPT. 
AT A HIGH LEVEL, WHAT IT MEANS IS 
THAT THE SERVICE THAT IS CONNECTING 
TO YOUR DATABASE DOESN'T ACTUALLY 
HAVE ANY INHERENT PERMISSIONS TO 
THAT DATABASE, IN PARTICULAR, IT 
DOESN'T HAVE THE MASTER KEY. AND 
THAT'S KIND OF HUGE. WITH A MASTER 
KEY, THAT'S IT, IT'S GAME OVER. 
IF YOU LOSE THAT, YOU CAN DO WHATEVER 
YOU WANT TO THE COSMOS DB, RIGHT? 
SO WHAT I'M NOT SHOWING HERE IS 
THE AUTHENTICATION PART BUT IT'S 
ASSUMED THAT YOU HAVE YOUR CLIENT 
THAT'S ALREADY BEEN AUTHENTICATED, 
MAYBE TO SOME KIND OF FRONT END 
AUTHENTICATION SERVICE, MAYBE IT'S 
A FEDERATED TO EVEN YOUR CUSTOMER'S 
ACTIVE DIRECTORY ENVIRONMENT, WHATEVER 
IT IS, THE EXPECTATION IS YOU END 
UP IDEALLY WITH SOMETHING LIKE A 
JSON WEB TOKEN OR AN OA, SOMETHING 
THAT GIVES YOU A SET OF CLAIMS, 
AND WHAT YOU TYPICALLY EXPECT TO 
SEE IN THOSE KIND OF CLAIMS ARE 
THE TENANT IDENTITY AND PROBABLY 
THE USER'S IDENTITY. IN FACT, IF 
YOU'RE WRITING A MULTI-TENANT SOLUTION 
AND YOU DON'T KNOW THE TENANT'S 
CONTEXT AT ALL TIMES IN YOUR PROCESSING, 
THAT'S PRETTY WORRISOME, RIGHT? 
SO THESE ARE PRETTY MUCH TABLE STAKES 
WHEN YOU HAVE A MULTI-TENANT SERVICE. 
SO IN THIS CASE I'M GOING TO START 
OUT WITH THE CLIENT HAS ALREADY 
AUTHENTICATED, IT'S GOT SOMETHING 
LIKE A JSON WEB TOKEN WITH THESE 
CLAIMS, AND THESE CLAIMS OBVIOUSLY 
GO TO YOUR SERVICE AND IT DOES THE 
VALIDATION OF THESE CLAIMS. NOW, 
WE'VE AUTHENTICATED AND EVERYTHING 
LOOKS GOOD. THE SERVICE NEEDS TO 
EITHER WRITE OR READ FROM THAT COSMOS 
DB BUT IT DOESN'T HAVE ANY ACCESS 
ON ITS OWN. WHAT IT DOES IS, IT 
ADDS THE DATABASE QUERY THAT IT 
WANTS TO DO AND IT SENDS BOTH OF 
THOSE OVER TO A HELPER SERVICE THAT 
WE CALL THE TOKEN BROKER SERVICE. 
AND THAT'S THE ONLY ONE THAT'S MENTIONED 
THAT HAS THE MASTER KEY. IN PRACTICE, 
THE MASTER KEY IS PROBABLY IN SOMETHING 
LIKE AZURE KEY VAULT, IT'S UP TO 
YOU HOW TO SECURE THAT, IT'S CERTAINLY 
IN MEMORY, THOUGH. THE TOKEN BROKEN 
SERVICE, WE RECOMMEND THAT IT REVALIDATES 
THOSE CLAIMS. AND THAT'S BECAUSE 
OF AT LEAST PRIVILEGED CONCEPT. 
WHAT IF THAT SERVICE WAS COMPROMISED? 
YOU DON'T WANT TO NECESSARILY TRUST 
THAT SERVICE. SO THE DUE DILIGENCE 
HERE IS TO REVALIDATE THOSE CLAIMS. 
AND HONESTLY IF YOU'RE USING SOMETHING 
LIKE JSON WEB TOKEN, THAT'S USUALLY 
PRETTY FAST, YOU CAN -- IT CAN BE 
DONE IN MEMORY. YOU'VE REVALIDATED 
THOSE CLAIMS NOW, AND WHAT YOU CAN 
DO IS, YOU CAN TAKE THAT TENANT 
CLAIM AND YOU CAN COMPARE THAT TO 
THE DB QUERY AND YOU CAN SEE IF 
THAT TENANT IS ALLOWED TO MAKE THAT 
DB QUERY. AS MENTIONED, WE ARE DOING 
PARTITION KEY ISOLATION FOR THE 
MOST PART, PER TENANT, AND SO IT'S 
QUITE EASY. IF ACNE -- IF TENANT 
ACME IS TRYING TO MAKE A DB QUERY 
THAT GOES TO PARTITION FABRY CANT, 
THAT'S A NO GO, THAT'S AN OBVIOUSLY 
CROSS-TENANT QUERY. YOU CAN DO THIS 
AT THE COLLECTION LEVEL, IF YOU 
DECIDE TO DO COLLECTION LEVEL TENANT 
ISOLATION, BUT IT MAKES IT REALLY 
SIMPLE TO SEE IF THAT QUERY IS TRYING 
TO MANIPULATE SOMEONE ELSE'S TENANT'S 
DATA. ASSUMING THAT THE TENANT IS 
MAKING A QUERY THAT THEY'RE ALLOWED 
TO MAKE. IT GENERAL WHAT'S I CALL 
AN ACCESS HASH. THERE'S ACTUALLY 
TWO DIFFERENT THINGS THAT COSMOS 
DB ALLOWS YOU AS -- WELL, A SET 
OF DATA THAT YOU CAN USE TO AUTHENTICATE. 
AND THAT GETS SENT BACK TO YOUR 
ORIGINATING SERVICE WHICH THEN USES 
THAT TO FINALLY MAKE THE REQUEST 
TO COSMOS DB AND SO FORTH. SO WHAT 
I'M GOING TO COVER NOW ARE THOSE 
TWO DIFFERENT TYPES OF ACCESS HASHES. 
EVERYONE IS FAMILIAR WITH A MASTER 
KEY, BUT THAT'S NOT ACTUALLY WHAT 
GOES OVER THE WIRE WHEN YOU AUTHENTICATE 
TO COSMOS DB. THAT WOULD BE BAD. 
UNDERNEATH THE HOOD WHAT ACTUALLY 
HAPPENS, AND THIS IS WELL DOCUMENTED, 
IS, IT GENERATES AN ACCESS SIGNATURE 
THAT BASICALLY IS A HASH OF YOUR 
REQUEST USING THE MASTER KEY. AND 
THAT IS WHAT I CALL THE MASTER KEY 
SIGNATURE. AND THAT IN TURN IS WHAT 
IS MENTIONED -- GOES OVER THE WIRE. 
IT'S ACTUALLY ONE PART OF THE AUTHORIZATION 
HEADER. THERE'S OTHER STUFF IN THERE 
THAT MAKES UP THE FULL AUTHORIZATION 
HEADER. NOW, LET'S TALK ABOUT RESOURCE 
TOKENS. NOT EVERYONE IS FAMILIAR 
WITH RESOURCE TOKENS BUT IT IS WELL 
SUPPORTED IN ALL OF THE VARIOUS 
DIFFERENT API INTERFACES TO COSMOS 
DB, AND UNDERNEATH THE HOOD WHAT 
ACTUALLY HAPPENS IS COSMOS IS CREATING 
A USER, A SPECIAL USER, WITH A SET 
OF PERMISSIONS. IT'S BASICALLY LIKE 
A DELEGATED MASTER KEY. EVEN THOUGH 
THIS ISN'T DOCUMENTED AT ALL IN 
MICROSOFT'S DOCUMENTATION, UNDERNEATH 
THE HOOD THEY ACTUALLY ARE GENERATING 
A RESOURCE TOKEN SIGNATURE AND THAT'S 
WHAT GETS SENT OVER THE WIRE. SO 
NOW LET'S TALK ABOUT WHAT YOU CAN 
ACTUALLY USE AND SEND BACK FROM 
YOUR TOKEN BROKER SERVICE. YOU DON'T 
WANT TO SEND BACK THE MASTER KEY, 
THAT'S THE WHOLE POINT. THE MASTER 
KEY SIGNATURE, ON THE OTHER HAND, 
IS GREAT. VERY DELEGATED TIGHT RESTRICTIONS, 
AND I'LL TALK ABOUT THE FEATURES 
OF THAT. THE RESOURCE TOKEN IS ALSO 
MEANT TO BE SOMETHING THAT YOU CAN, 
YOU KNOW, SEND OVER THE WIRE TO 
ANOTHER SERVICE. THE RESOURCE TOKEN 
SIGNATURE IS NOT DOCUMENTED ON PURPOSE 
TO BE FAIR, RIGHT? IT'S MORE OF 
A BLACK BOX. AND UNLESS YOU REALLY 
WANT TO REVERSE ENGINEER THAT, I 
DON'T WANT RECOMMEND SENDING THAT. 
YOU COULD SEND THE AUTHORIZATION 
HEADER, BUT THE SQL INTERFACES DON'T 
REALLY PROVIDE GOOD SUPPORT FOR 
INTERACTING WITH AUTHORIZATION HEADERS. 
SO IT'S KIND OF RECOMMENDED THAT 
YOU SEND BACK THE RESOURCE TOKEN, 
WHICH AS MENTIONED HAS GOOD SUPPORT, 
OR SEND BACK THE MASTER KEY SIGNATURE 
IF THAT'S THE PART THAT YOU'RE USING. 
NOW, LET'S TALK ABOUT THE DIFFERENCES 
BETWEEN THE TWO AND THE PROS AND 
CONS BETWEEN THE TWO BECAUSE THERE'S 
REALLY SOME KEY THINGS THAT YOU'VE 
GOT TO BE CAREFUL ABOUT. THE MASTER 
KEY SIGNATURE IS SUPER FAST TO GENERATE, 
IT'S JUST A HASH. IT'S ALL DONE 
IN MEMORY. IT'S ONLY VALID FOR A 
FEW MINUTES, WHICH IS ACTUALLY A 
GOOD THING. AND IT'S VERY SPECIFIC 
TO THE OPERATION THAT YOU WANT TO 
DO. IF YOU WERE TRYING TO EDIT, 
IT'S GOING TO ONLY LET YOU EDIT, 
IT'S NOT GOING TO ALLOW YOU TO CREATE 
OR DELETE OR ANYTHING LIKE THAT. 
THAT'S REALLY USEFUL IF YOU'RE TRYING 
TO DO LIKE A LEAST PRIVILEGED PRINCIPLE. 
BUT THIS IS THE KILLER AND VERY 
UNFORTUNATE. IT DOESN'T ENCODE THE 
PARTITION KEY THAT IT WAS TARGETING. 
AND SO IF YOU GET -- IF YOU HAVE 
A MASTER KEY SIGNATURE, IT CAN BE 
USED FOR ANY PARTITION KEY, AND 
IF YOU'RE DOING ISOLATION ON A PER 
PARTITION BASIS LIKE WE ARE, THAT 
WAS A NO GO FOR US. IF YOU'RE DOING 
IT ON A COLLECTION BASIS, HOWEVER, 
THIS IS GREAT. SO LET'S TALK ABOUT 
RESOURCE TOKENS NOW, AND THIS IS 
ACTUALLY WHAT WE ENDED UP USING. 
THE DOWN SIDE WITH THE RESOURCE 
TOKEN IS YOU CAN'T GENERATE IT YOURSELF. 
IT HAS TO GO TO THE COSMOS DB. AND 
THEN YOU HAVE THAT ROUND TRIP TIME, 
THE ACCESS TIME. AS MENTIONED UNDERNEATH 
THE HOOD IT'S CREATING A USER, CREATING 
PERMISSIONS AND SO FORTH. IT'S GOT 
AT VERY FLEXIBLE EXPIRATION TIME. 
THE DEFAULT IS FOUR HOURS BUT YOU 
CAN HAVE IT UP TO A DAY. AND THAT 
DOES LEND ITSELF TO SOME NICE THINGS. 
YOU CAN CACHE THESE RESOURCE TOKENS. 
SOME DOWN SIDES, THE LONGER YOU 
CACHE IT, THE LONGER THE EXPIRATION, 
THE LONGER THE DAMAGE A MALICIOUS 
PERSON COULD DO, BUT IT MAKES IT 
CONVENIENT AND ALSO REDUCES THE 
DOWN SIDE, WHICH IS THAT ROUND TRIP 
TIME TO GO GET IT. AND THE SCOPING 
IS EITHER READ OR ALL. YOU DON'T 
GET THAT SUPER FINE LEVEL GRANULATOR 
THAT YOU DID WITH MASTER KEY SIGNATURE. 
I'M GOING TO SHOW YOU NOW THE LATENCY, 
A DEMO OF WHAT THIS MEANS IN PRACTICE. 
AND IN PARTICULAR, WHAT I'M GOING 
TO SHOW YOU IS THE LATENCY BETWEEN 
YOUR FRONT END SERVICE, WHEN IT 
MAKES A CALL TO THE TOKEN BROKER, 
AND THEN YOU GET BACK THAT ACCESS 
HASH. AND THAT'S EITHER THE MASTER 
KEY SIGNATURE OR IT'S THE RESOURCE 
TOKEN. SO I'M GOING TO SWITCH NOW 
TO MY LAPTOP . IN THE TOP HERE, 
IT'S NOT VERY OBVIOUS, BUT IN THE 
TOP HERE IS MY TOKEN BROKER, AND 
I'M ACTUALLY DOING A REMOTE LOG-IN, 
I'M LOOKING AT THE LOGS FOR A WEB 
APP FOR CONTAINER I HAVE, APPARENTLY 
FOR THE LAST 15 MINUTES THERE'S 
BEEN NO TRACES. ON THE BOTTOM ONE, 
IS MY CONFIGURATION SERVICE, MY 
FRONT END SERVICE THAT ACTUALLY 
IS THE ONE MAKING THOSE DATABASE 
REQUESTS TO THE COSMOS DB. AND THEN 
I'M JUST GOING TO USE POSTMAN TO 
SIMULATE A CLIENT REQUEST. AND I'M 
GOING TO START WITH THE RESOURCE 
TOKEN. AND I'M GOING TO MAKE SOME 
REQUESTS AND I'M GOING TO MAKE A 
COUPLE OF THEM JUST SO WE CAN SEE 
AVERAGES. AND YOU CAN SEE THAT I 
AM GETTING BACK SOME TENANT INFORMATION. 
NOW, I'M NOT SEEING ANY TRACES. 
WHAT I'M GOING TO DO IS, I'M GOING 
TO TERMINATE THIS, I THINK IT WAS 
JUST A LITTLE BIT TOO LONG. WE'RE 
GOING TO CONNECT BACK AND DO THAT 
REMOTE LOGGING. IN FACT, YOU CAN 
SEE THE TRACE MESSAGES THERE THAT 
WEREN'T SHOWN EARLIER. YOU CAN SEE 
THE ROUND TRIP TIME FROM THE TOKEN 
BROKER SERVICE TO COSMOS DB TO GENERATE 
A RESOURCE TOKEN IS TAKING AROUND 
55 TO 60 MILLISECONDS OR SOMETHING 
LIKE THAT. NOW, THAT IS CONTROL 
PLANE, COSMOS DB IS DOING SOME WORK 
THERE WHICH IS WHY YOU SEE THOSE 
DOUBLE-DIGIT NUMBERS, AND THEN FROM 
THE -- YOUR CONSUMING SERVICE, THAT'S 
TYPICALLY AROUND 150 MILLISECONDS 
OR SO, ASSUMING THAT THEY'RE IN 
THE SAME, YOU KNOW, CLOSELY LOCATED. 
SO BASICALLY IT'S AN EXTRA LATENCY 
OF 150 MILLISECONDS TO GET THAT 
REALLY NICE TENANT ISOLATION AND 
TO BASICALLY REMOVE THE POSSIBILITY 
OF SQL STYLE INJECTION ATTACKS, 
WHICH CAN ABSOLUTELY HAPPEN WITH 
SOMETHING LIKE A NO SQL SOLUTION, 
SUCH AS COSMOS DB. SO THAT'S BEEN 
VERY ACCEPTABLE FOR US AND, AGAIN, 
THIS IS WITHOUT CACHING. AND HONESTLY, 
IT'S WITHOUT ANY OPTIMIZATION WHATSOEVER. 
NOW LET'S GO TO MASTER KEY SIGNATURES. 
I'M GOING TO MAKE THIS REQUEST, 
A COUPLE TIMES AGAIN, AND IT IS 
SO FAST THAT IT'S SHOWING UP AS 
ZERO MILLISECONDS. AGAIN, ALL THAT 
IT'S DOING IS IT'S JUST HASHING 
THE REQUEST IN A SPECIFIC WAY THAT 
WORKS AND IS UNDERSTOOD BY COSMOS 
DB. SO THIS IS, AS MENTIONED, BY 
FAR THE SUPERIOR SOLUTION IF YOU'RE 
NOT DOING PARTITION KEY ISOLATION. 
AND THEN IF YOU LOOK AT THE TOTAL 
ROUND TRIP TIME, THE TOTAL LATENCY 
FROM THE POINT OF VIEW OF SERVICE 
THAT'S MAKING THE WRITES ON THE 
CUSTOMER'S BEHALF, IT'S AROUND ABOUT 
50 MILLISECONDS OR SO, JUST REALLY 
NICE. THAT KIND OF CONCLUDES THE 
CONCEPT OF THE TOKEN BROKER, YOU 
KNOW, ARCHITECTURE PATTERN, AND 
HOW WE APPLY WHAT YOU LEARNED FROM 
ANDREW AND MARK IN PRACTICE AT CITRIX, 
IN PARTICULAR IN THE END-POINT MANAGEMENT 
GROUP. THANKS. PASS IT BACK NOW. 
[ APPLAUSE ] >> OKAY, SO I GUESS 
SOME SESSION TAKE-AWAYS, WE JUST 
WANT TO MAKE SURE YOU GUYS KIND 
OF WALK AWAY WITH, YOU KNOW, WHAT 
WE WANT YOU TO UNDERSTAND AROUND 
HERE. I WANT TO DO SOME POLLING. 
AFTER LISTENING TO OUR TALK, WHO 
UNDERSTANDS KIND OF BETTER THE PERFORMANCE 
ISOLATION VERSUS COST WHEN BUILDING 
MULTI-TENANT SYSTEMS? YES? AWESOME. 
HOW ABOUT THE TRADEOFFS FOR CONSISTENCY, 
AVAILABILITY, LATENCY, THE CAP THERM, 
PACK LC? ON THE SECURITY ISOLATION, 
TOKEN BROKER, DO YOU HAVE A GOOD 
GRASP OF THAT, SHOW OF HANDS? [ 
LAUGHTER ] FIVEs ACROSS THE BOARD 
THEN? YES, ALL THE WAY ACROSS? YAY. 
YES. SO -- >> THANK YOU, GUYS. >> 
SO JUST SOME RESOURCES HERE, SO 
ALL THE DEMOS I SHOWED AVAILABLE 
ON GitHub, SO IF YOU WANT TO TAKE 
A SNAPSHOT OF THAT GO AHEAD, GO 
AHEAD AND DOWNLOAD THIS. THERE'S 
A COUPLE OTHER DEMOS IN THERE, TOO. 
I'LL SHOW YOU, I HAVE A TEST IN 
THERE THAT SHOWS CONFLICT RESOLUTION 
FOR DOING MULTI-MASTER, SO THAT 
WILL SHOW LAST WRITER WINS AS WELL 
AS HOW TO USE A MERGE PROCEDURE, 
ALSO A FUN ONE WHERE I SHOW YOU 
CUSTOM SYNCHRONIZATION TO HELP CHEAT 
SPEED OF LIGHT. AND TERRY, HIS TOKEN 
BROKER PRODUCT WITHIN HERE THAT 
HE SHOWED AS WELL, TO HELP YOU GET 
FROM ZERO TO 100 ON IMPLEMENTING 
THAT AS A PATTERN WITHIN YOUR APPLICATION. 
SO WE'LL TAKE SOME QUESTIONS WHILE 
WE'VE GOT LIKE 30 SECONDS. THANK 
