EVERYONE. MY NAME IS BUCK WOODY. 
AAPPLIED DATA SCIENTIST. >> I'M 
MIHAELA BLENDEA. I WORKED ON CLUSTER. 
WE ARE HEAR TO TALK ABOUT THAT. 
ASK WE WILL TALK ABOUT THE -- WE 
WILL TALK ABOUT ARCHITECTURE. ONE 
OF THE THINGS I TRY TO DO IS TO 
MAKE IT REAL. WE HAVE TO HAVE A REAL LIVE 
SCENARIO. WE WILL JUMP INTO THE TECH AND 
DIVE DEEP BUT WE WILL SPEND 
A BIT OF TIME TALKING ABOUT WHY WE
WOULD DO THIS. 
SO WHAT IS BIG DATA?
YOU DIDN'T 
KNOW I WAS GOING TO 
ASK YOU THAT. 
>> I'M SURE YOU WILL TELL US. >> 
I AM. HOW ABOUT THIS? IS A TERABYTE 
BIG DATA? PETABITE IS BIG DATA. 
HOW ABOUT AN EXO BITE? WHAT ABOUT 
THAT? ZEBA BYTE. WHAT IS AFTER THAT? 
>> A YOTTABYTE. WHAT IS BIG DATA? 
HERE IS THE DEFINITION. BIG DATA 
IS NEW DETERATA YOU CANNOT PROCESS 
IN THE TIME YOU WANT WITH THE TECHNOLOGY 
YOU CURRENTLY HAVE. THAT IS IT. 
NO MORE COMPLICATED THAN THAT. IF 
YOU HAVE COMMODORE 64, ONE MEGABYTE 
IS BIG DATA. THERE IS BIG DATA IN 
THE WORLD. WHEN WE THINK ABOUT THIS, 
THE REASON THERE IS BIG DATA, IS 
THAT WE HAVE HAD LONGER TO DO IT. 
AND IT'S BEEN CHEAPER TO DO IT RIGHT? 
HOW MANY HAVE THAT KITCHEN DRAWER? 
GOT THAT KITCHEN DRAWER IN YOUR 
HOUSE AND YOU CAN'T OPEN IT ANYMORE? 
AND WHY DO YOU HAVE THAT? IT HAS 
GOT A SPATULA, YOU CAN'T OPEN IT. 
SOMETHING IS HOLDING IT CLOSED. 
YOU JUST SHOVE STUFF IN THERE, THERE 
IS WARRANTYS OF THING YOU DON'T 
EVEN OWN. SAME THING WITH BUSINESS 
DATA. THERE IS A LARGE RETAILER 
WITH A BLUE AND YELLOW LOGO HERE 
IN THE UNITED STATES OF AMERICA 
THAT TAKES IN 1. 5 PEDABYTES OF 
DATA AN HOUR. THAT IS LOTS OF DATA. 
THEY HAVE FIVE FULL SYSTEMS THAT 
PROCESS THEIR BIG DATA. I DON'T 
DO WORDY SLIDE BUT THESE ARE INDUSTRIES 
I CURATED. THESE ARE THE NUMBER 
ONE TWO, THREE, FOUR AND USERS OF 
BIG DATA. HERE IS THE USED CASES 
ON THE RIGHT OF WHAT THEY ARE DOING. 
YOU CAN SEE INSIDE RETAIL, DEMAND 
PREDICTION. HOW MANY PEOPLE IN HEALTH 
CARE? YOU WILL FIND IT INTERESTING 
THAT THE NUMBER ONE USE CASE FOR 
BIG DATA IN HEALTH CARE IS NOT CURING 
PEOPLE BUT IN GETTING PAID. I KNOW 
THAT WILL BE SHOCK TO EVERYBODY 
HERE. SO THE POINT IS, THERE IS 
BIG DATA AND THERE IS REASONS TO 
USE BIG DATA, RIGHT?  SO THERE IS 
ONLY ONE WAY TO DO THIS. ONLY ONE 
WAY TO PROCESS BIG DATA. HOW MANY 
OF YOU KNOW WHAT SCALE UP MEANS? 
MAKING THE BIG FOUR BIGGER. C. P. 
U. , MEMORY, DIX AND NETWORK, YOU 
KEEP SHOVING MORE AND MORE BUT AT 
SOME POINT YOU RUN OUT OF THE ABILITY 
TO DO SOMETHINGS IN A PROCESSING 
WAY BECAUSE OF AN OTHER THINGS. 
SCALE OUT PROCESSING, THE WAY TO 
FIX THIS. IF YOU HAVE BEEN TO A 
GROCERY STORE YOUNG HAVE DEALT WITH 
SCALE OUT PROCESSING. THERE IS REGISTERS 
AND SOMEBODY IS AT THE REGISTER 
AND GOT AN ALGORITHM, TAKE THE PRIVATES 
YOU ITEM AND -- THE PREVIOUS ITEM 
AND IT IS AN ALGORITHM. YOUR DATA, 
TOILET PARENT AND MAGAZINE OR BOSCH 
OR WHATEVER YOU ARE BUYING IS THE 
DATA. WHEN THE LINE GETTING LONG, 
WHAT DO THEY DO? >> MORE PEOPLE. 
THEY ADD ANOTHER REGISTER. RIGHT? 
AND THAT REGISTER KNOWS THOUSAND 
DO THE EXACT SAME CALCULATION. THEY 
KNOW. BUT THEY HAVE DIFFERENT ITEMS 
GOING THROUGH THAT LINE. THAT IS 
DIFFERENT DATA. THERE IS ALWAYS 
THAT LADY THAT HAS THE BAND ON HER 
WRIST WITH THE KEY AND SHE IS THE 
ONLY ONE THAT CAN TELL PEOPLE THAT 
IT IS YOUR TURN ON THE REGISTER, 
GO TAKE YOUR BREAK. KNOW WHAT I'M 
TALKING ABOUT? ALWAYS. WELL, GUESS 
WHAT? YOU JUST LEARNED HADOOP. YOU 
CAN PUT THAT ON YOUR RESUME. LET 
ME PROVE IT. SEE THE H. D. F. S. 
? THAT IS DISTRIBUTED DATA. THAT 
IS YOU WITH YOUR TOILET PAPER AND 
YOUR BOSCH, RIGHT? AND THEN THE 
MAP REDUCE, THE MAP IS THE DIFFERENT 
REGISTERS. THEY HAVE THE SAME CALCULATIONS 
BUT THEY ARE WORKING ON DIFFERENT 
DATA. REDUCED PART IS AT THE END 
OF THE DAY, I TAKE THE FOUR REGSTERS 
AND TOE TELL THEM TOGETHER AND PUT 
ALL MY SALES BACK TOGETHER AND YARN, 
THAT THE LATEST WITH THE KEY. SERIOUSLY. 
SO IF YOU EVER USED A GROCERY STORE, 
YOU KNOW HOW TO USE HADU. IT IS 
GREAT BUT BATCH ORIENTED. KIND OF 
SLOW. WE NEED TO GET TO A FASTER 
WAY TO DO THIS. SO THE WAY WE DO 
THIS IS SOMETHING CALLED SPARK FROM 
OUR GOOD FRIENDS IN BERKELEY, CALIFORNIA. 
THEY TAKE THAT SCALE OUT STORAGE, 
THE BLUE THING AT THE BOTTOM. THEY 
HAVE NODES, AND LOAD THINGS UP BUT 
SPARK REALLY, IF YOU JUST SIMPLIFY 
SPARK, ALL IT IS DOING A TURNING 
TO THE REDUNDANT DATA INTO AN OBJECT 
AND DOING COMPRESSION AND MEMORY 
STUFF. THEN THERE ARE LIBRARIES 
THAT CAN WORK ON IT.  THE FIRST ONE IS CALLED 
SPARK.  CAN LOAD THINGS INTO A RESILIENT 
DATA SET. IF YOU WANT TOO WORK IN 
A TAP YOU LAR WAY -- TABULAR FRAME. 
YOU WANT TO WORK WITH PYTHON OR 
SCALA YOU TURN IT INTO ANOTHER ING 
THAT IS ALL THAT SPARK DOES. MAKING 
THE DISTRIBUTING SCALING PROCESSING 
FASTER? MAKE SENSE? YOU HAD A 30 
SECOND OVER VIEW ON HOW YOU SCALE 
PROCESSING IN STORAGE. WHAT DO WE 
KNOW?  >> BIG DATA, NEED TO PUT 
IT ON SOMETHING. WE BUY A COME PAUTER. 
INSTALL AND OPERATING SYSTEM LIKE 
MICROSOFT WINDOWS AS THE GOOD LORD 
INTENDED AND INSTALL THINGS ON IT 
AND THEN WHAT WE DO IS GET SOMETHING 
CALLED THE HYPER VISOR. IT DOES 
TWO THINGS. I WANT CREATES A FAKE 
HARD DRIVE, AND IT CREATES A FAKE 
BIOS. AND THE BIOS REREPRESENTS 
C. P. U. MEMORY WORKING IN DISC. 
AND THEN YOU INSTALL AN ON IT RAG 
SYSTEM LIKE WINDOWS, AND THEN NO, 
THE IT'S ALL GOOD. YOU CAN PUT RED 
HAT ON THERE IF ONTARIO WANT TO. 
WE LOVE YOUR PARTNERS THERE AND 
ALL OF THAT IS JUST SO WE CAN RUN 
PYTHON AND CODE. THAT IS ALL WE 
DO. THIS IS WHAT WE TO TODAY. HOPEFULLY 
THAT WASN'T A REFRESHER. YOU KNEW 
THAT ALREADY. IF I WANT MORE, I 
KEEP ADDING THEM BUT THEY CAN GET 
KIND OF BIG. I'M ALSO CARRYING THE 
OPERATING SYSTEM EVERY TIME. THE 
MOUSE, DRIVER, VIDEO, ALL THE STUFF 
I DON'T REALLY NEED. THIS IS NOT 
THE GREATEST WAY TO DO OUR DISTRIBUTIVE 
PROCESSING. IT'S CHEAPER THAN BUYING 
PHYSICAL HARDWARE, BUT IT IS STILL 
FAIRLY HEAVY. IS EVERYBODY WITH 
ME SO FAR? GOOD. LET'S MOVE ON NOW 
AND START TALKING ABOUT THE FOUNDATION 
OF BIG DATA CLUSTER.  WE KNOW SPARK, 
VIRIZATION, SNOW LET'S MOVE TO CONTAINERS. 
WHAT WE DID WAS SAID, WE ALREADY 
HAVE C. P. U. DISC AND MEMORY AND 
NOTE WORKING -- NETWORKING WHY DON'T 
WE JUST USE IT? THERE IS AN OPERATING 
SYSTEM. USE THAN WE DO. YOU CREATE 
A FILE. A THING CALLED THE CONTAINER 
RUN TIME. THE ONE WE USE IS DOCKER 
IT IS A DAMON AND RUNNING IN THE 
BACK GROUND AND WHAT IT CAN DO IS 
IT CAN TAKE A FILE, A TEXT FILE, 
THAT SAYS I WOULD LIKE PYTHON, MY 
BINARIES AND SOME CODE, JUST A DESCRIPTION, 
A TEXT FILE. YOU THEN RUN SOMETHING 
CALLED A COMPOSE AND IT WILL BUILD 
AN IMAGE, A BUY MARE THING THAT 
HAS -- BINARY THING, LIKE A ZIP 
FILE REALLY AND ALL THAT IS IN THERE. 
THEN WHEN YOU DO A DOCKER RUN, THOSE 
THINGS STAND UP IN THE MEMORY SPACE. 
THEY ARE CARVED OUT, THEY ARE SAFE 
AND NOW IT IS RUNNING. BUT YOU WILL 
NOTICE I'M NOT CARRYING VIDEO DRIVER 
ANSWER SERVICE PACKS AND ALL THAT. 
MUCH LIGHTER. MUCH LIGHTER. BUT 
IT GETS EVEN BERT WITH THIS. -- 
BETTER WITH THIS. ONCE WE HAVE THESE, 
WE CAN SPIN UP LOTS WITH A COMMAND. 
WE HAVE DEFINED THESE. YOU KNOW 
IN THE SQL, LANGUAGE, I WANT THE 
FILES AND DATA AND THE SYSTEM FIGURES 
OUT HOW TO DO THAT. WITH A CONTAINER 
YOU HAVE DECLARATIVE COMPUTER. I 
WANT PIE TYTHON THREE SEVEN AND 
IT FIGURES IT OUT. THE WE -- WE 
ALSO WANTED PYTHON TWO FIVE, THAT 
IS THE CONTAINER ON THE RIGHT. WHAT 
ABOUT THE MIDDLE ONE? HERE IS THE 
INTERESTING THING. WHAT YOU CAN 
DO INSIDE A CONTAINER TECHNOLOGY 
IS IF YOU ASK FOR PYTHON THREE FIVE 
AGAIN, IN THE IMAGE, IF IT IS ALREADY 
RUNNING IN MEMORY THE SHARES IT. 
IT'S IN LAYERS. SO THIS IS AWESOME. 
SO THIS IS GREAT WAY TO DISTRIBUTE 
THINGS. >> DON'T WORRY, WE WON'T 
ASK YOU TO BUILD CONTAINERS. >> 
NO. HOW ABOUT IF WE DO IT. LET'S 
-- WE WILL DO IT. MICROSOFT WILL 
DO IT FOR YOU. WAP DO WE DO WHEN 
SOMETHING IS EASY TO DO? WE DO A 
LOT OF IT. THERE IS CAKE IN THE 
BREAK ROOM. YOU GO EAT CAKE BECAUSE 
THERE IS CAKE IN THE BREAK ROOM. 
SO THESE THINGS END UP SPRAWLING 
ALL OVER THE PLACE. WE HAVE A PROBLEM 
WITH STORAGE. THE STORAGE IS IN 
IT. IF THE CONTAINER GOES AWAY, 
I LOST MY STORAGE. THAT IS NOT GOOD. 
>> AND THAT IS DATA. STORAGE IS 
AN IMPORTANT. >> IT IS A DATA BASE. 
SO WHAT DO WE DO? WE SOLVE IF ONE 
ABSTRACTION PROBLEM WITH ANOTHER 
ABSTRACTION AND CALLED CONTAINER 
ORCHESTRATION. IT IS CALLED KUBERNETES. 
THAT MEANS PILOT BUT IT WAS TOO 
LONG SO THEY SHORTENED TO KEIGHTHS. 
IT IS NO EASIER TO STAY. THEY NAMED 
IT. WHY DIDN'T THEY PICK A SHORT 
NAME TO BEGIN WITH? I DON'T KNOW. 
I DEGREES. I DIGRESS. >> WE WILL 
TALK ABOUT IT. >> SO WE KNOW ABOUT 
CONTAINERS. FIRST CONCEPT IN KUBERNETES 
IS THE IDEA OF SOMETHING CALLED 
A POD.  A POD IS A WRAP AROUND ONE 
OR MORE CONTAINER. THREE CONTAINERS 
RUNNING IN THE POD AND KUBERNETES 
SAYS IT WILL HANDLE THAT AND IT 
CAN MOVE THEM AROUND. THOSE LIVE 
ON A NODE, WHICH IS COMPUTER OR 
VIRTUAL P. C. THE NODE AS THIS THREE 
THINGS IN IT, DOCKER, HAS AN ORCHESTRATION 
SHIM, THE COMMAND THAT LETS IT KNOW 
IT IS PART OF KUBERNETES AND I. 
P. SHIM THAT IS A MINIATURE IP CORD, 
JUST A PORT DIRECTOR. YOU PUT A 
BUNCH TOGETHER. YOU MAKE A MASTER 
NODE. AND YOU GET A CLUSTER. THAT 
IS KUBERNETES IN A MINUTE AND A 
HALF. MAKES SENSE?  WE STILL HAVE 
A PROBLEM HERE. >> STORAGE?  >> 
STORAGE. WE DO A VOLUME AND VOLUME 
IS A SET OF STORAGE THAT IS REDUNDANT, 
IT'S FAST, HIGHLY AVAILABLE, IT'S 
SOMEWHERE ELSE, AND WE THEN TELL 
THE POD, YOU HAVE ACCESS TO THAT 
STORAGE, VIA SOMETHING CALLED THE 
CLAIM. YOU CAN THINK OF THIS, CONCEPTUALLY, 
ALTHOUGH NOT ACCURATE, THINK OF 
THIS LIKE A SAND WIRE, IF YOU WANT. 
HERE IS WHAT IS INTERESTING. IF 
THE POD GOES DOWN, AND KUBERNETES 
WAYNES UP ON ANOTHER NODE -- WAKES. 
IT UP ON ANOTHER NODE, IT WILL REDIRECT 
THE CLAIM TO GO TO THE RIGHT PLACE 
AND YOU WON'T KNOW. SO IT LOOKS 
LIKE WE HAVE SOLVED THE PROBLEMS. 
WE HAVE GOT BIG DATA. A REASON FOR 
BIG DATA, A WAY TO PROCESS BIG DATA 
AT SCALE AND NEW WE HAVE GOT A WAY 
TO DISTRIBUTE BIG DATA. PROCESSING 
SYSTEMS AND WE HAVE GOT A WAY TO 
CONTROL THAT SPRAWL. WE HAVE GOT 
EVERYTHING WE NEED, BUT WE HAVE 
A PROBLEM. >> WE NEED THE CLUSTER. 
>> NOT ONLY, THAT SEQUEL RUNS ON 
WINDOWS. SO WE REROUTE IT. ACTUALLY, 
WE DIDN'T. WE TOOK A SHIM THAT SITS 
IN BETWEEN LYNN LYNN EX AND SEQUEL 
SERVE PER SO THE INTERESTING THING 
IS THAT THE CODE THAT IS RUNG. -- 
INTERESTING STUFF.  SO I CAN NOW 
RUN AN E. X. E. LYNN NUK, WHICH 
YOU CAN'T DO. >> WE HAVE ALL THE 
INGREDIENT. PUT IT TOGETHER. GO 
THROUGH THE PIECES AND SHOW US THINGS. 
HERE WE GO O. P. P. THIS IS -- WHEN 
I BUILD OUT THIS SLIDE, I WILL LET 
YOU KNOW IF YOU WANT TO TAKE YOUR 
CAMERA PICTURES. THE KUBERNETES 
CLUSTER.  . SO WHAT WE DO, USE ONE 
SIMPLE COMMAND, GRAB ANOTHER MARKUP 
FILE AND BEGIN TO BUILD THINGS AND 
THE FIRST THING WE BUILD IS SOMETHING 
CALLED A CONTROLLER. A CONTROLLER. 
AND THIS CONTROLLER IS SOMETHING 
WE ROUTE AND IT IS A SERVICE AND 
A POD AND A NODE ON A CLUSTER AND 
THAT WAITS FOR US TO TELL IT OTHER 
THINGS TO DO. IT WEIGHS UP, DEPLOYS 
ITSELF AND SAYS OK. SO THIS IS JUST 
A CONTAINER. BUT THEN IT SAYS I 
NEED SOME HELP. I NEED TO BE ABLE 
TO MONITOR. I NEED TO BE ABLE TO 
MANAGE. I NEED TO BE ABLE TO AUTHENTICATE, 
ACTIVE DIRECTORY. I NEED ALL KINDS 
OF STUFF. I WILL JUST CALL THIS 
SHARED SERVICES FOR NOW AND PUT 
IT IN -- EVERYBODY GOOD SO FAR IN 
ALL RIGHT. THE NEXT THING THAT THE 
CONTROLLER DOES, IS HE THROWS DOWN 
SEQUEL SERVICER IN A CONTAINER RUNNING 
IN LINUX, IN A POD, IN A NODE IN 
A CLUSTER. IT IS A SEQUEL. IT IS 
JUST SEQUEL, RIGHT? AND YOU CAN 
EVEN USE THINGS LIKE POLY BASIC 
THE ABILITY FOR US TO QUIRKY ORACLE 
OR HDFS, INCLUDES THAT BUILT IN, 
READY TO GO. THIS IS NICE BUT STILL 
NOT SCALED, IS IT? >> IT IS NOT. 
YOU STILL DON'T SEE -- >> YOU DON'T SEE 
THE OTHER STUFF I TALKED ABOUT.  
STICK 
WITH ME. WHAT IF WE WERE TO TAKE 
ANOTHER SET OF SEQUEL SERVERS THAT 
YOU DON'T TALK TO.
YOU TALK TO THE 
MASTER OF SEQUEL SERVERS IT TALKS 
TO OTHER SERVERS AND SCALED OUT 
TO POLY BASE. I DID SAY SOMETHING 
ABOUT BIG DATA. THE P. D. W. , PARALLEL 
DATA WAREHOUSE FOR MICROSOFT IN 
TOOK A LOT OF THE ENGINE PART, A 
PARALLEL SEQUEL ENGINE. WE HAVE 
GOT ONE OF THOSE. WE MADE IT INTO 
SOFTWARE, AND ONCE AGAIN, NOW, YOU 
CAN TALK TO THE MASTER INSTANCE 
AND WE WILL SCALE AND SHARD THOSE 
DATA BASES THROUGH A SEQUEL DATA 
POOL. BUT WAIT, THERE IS MORE. WE 
TALKED A BITN'T THAT SPARK THING 
-- A BIT ABOUT THIS SPARK THING. 
WE WILL STAND THIS UP AND CON FIGURE 
IT FOR YOU. AND NOT ONLY THAT, WE 
WILL PUT ANOTHER SEQUEL SERVER YOU 
TALK TO, THAT CAN TALK TO HDFS DIRECTLY. 
NOW YOU CAN TALK TO SEQUEL, ORACLE, 
EXTERNAL HDFS COSMO DB HUGE DATA 
POOLING TO SPARK AND HDFP ALL AT 
ONCE. THERE IS MORE. >> IT IS SMART? 
>> IT IS SUPER SMART. WATCH THIS. 
SO SOME FOLKS ON THE TEAM, SAT DOWN 
AND SAID, KNOW WHAT WOULD BE COOL 
IS IF ONTARIO CAN MOUNT AMAZON S-3 
OR A. D. L. S. , VERSION TWO, AS 
A MOUNT POINT ON H. D. F. S. SO 
THAT YOU CAN SELECT FROM A TEXT 
FILE DIRECT TOIR WITH THOUSANDS 
OF FILES IN IT AS IF IT WERE A TABLE 
INSIDE YOUR BUILDING. AND JOIN THAT 
TO ORACLE AND JOIN THAT TO SEQUEL. 
SO WE DID THAT. THAT IS CALLED H. 
D. F. S. TIERING, TIERING. BUT WAIT, 
THERE IS MORE. >> IT'S SMART. >> 
VERY SMART.  WE HAVE GOT SUZIE THE 
SCIENTIST AND SHE NEEDS TO DO SOME 
WORK SO WE HAVE DONE A LOT OF TRAINING 
AND NELLY AND I WILL SHOW YOU AN 
EXAMPLE OF THAT TRAINING OF MACHINE 
LEARNING MODEL ON BIG DATA. >> RUN 
THOSE INSIDE. >> RUN THOSE IN BIG 
DATA CLUSTER. WE CAN USE AN APP 
PEOPLE WHERE YOU CAN DEPLOY THING, 
MORE PODS, THAT ARE THINGS LIKE 
MACHINE LEARNING SERVE OTHER FLASK 
APP IN PYTHON OR SSIS RUNNING JOBS. 
AND ONCE AGAIN, THE PERSON HITS 
IT. HERE IS THE THING. DO YOU SEE 
THE BIG CIRCLE AROUND EVERYTHING? 
THAT IS KUBERNETES. WE DON'T CARE 
WHERE THAT IS. WE SUPPORT IT ON 
PRIM, IN AZURE, KUBERNETES SERVICE, 
ON OPEN SHIFT, AND OTHER DISTRIBUTES 
AS WELL OF KUBDZ, KUBERNETES. IF 
YOU ARE BANK, MILITARY, HOSPITAL 
YOU CAN KEEP IN THE BUILDING. IF 
YOU ARE CLOUD READY YOU CAN DO INSIDE 
AMAZON OR MICROSOFT. MAKE SENSE. 
OK. >> YOU TELL ME WHEN TO SWITCH 
YOUR SLIDES. >> THAT WAS A LOT. 
>> A LOT OF COMPONENTS. WE ARE GOING 
TO GO THROUGH HOW EXACTLY YOU ARE 
GOING TO MANAGE THOSE THINGS TOGETHER 
AS A UNIT. RIGHT? SO I WILL START 
BY GOING BACK TO WHAT YOU SAID EARLIER. 
SO LET'S NOT FORGET. SEQUEL SERVER 
BIG DATA, DEMROINED AS LINUX BASED 
CONTAINERS. WHY DID WE DO THAT? 
THIS CHOICE OF ARCHITECT CHUR COMES 
WITH A LOT OF ADVANTAGES THAT WE 
HAVE FOR BIG CLUSTER DEPLOYMENT. 
OK. PRETTY FAST. YOU CAN HAVE AN 
ENVIRONMENT UP IN A FEW MINUTES. 
THEN ALL THE SOFTWARE THAT IS REQUIRED, 
IS CONTAINED IN THE IMAGES THAT 
ARE BEING PULLED DOWN DURING THE 
DEPLOYMENT, SO ONCE THAT COMPLETES, 
YOU DON'T NEED TO INSTALL ADDITIONAL 
SOFTWARE OR DO ANYTHING. YOU CAN 
IMMEDIATELY START TO RUN SEQUEL 
SERVER WORKLOAD OR SPARK OR EVEN 
MANAGE THE ENVIRONMENT ITSELF. AND 
THEN, RELATED TO THE DEPLOYMENT, 
YOU HAVE UPGRADES. WE WILL MOVE 
ALONG, GOING TO ADD NEW VERSIONS 
OF THE SOFTWARE, AND YOU ARE MOST 
LIKELY GOING TO WANT TO STAY UP 
TO DATE AND GET THE LATEST AND GREATEST. 
AND THE ONLY THING YOU WILL HAVE 
TO DO AT THAT TIME IS TO JUST USE 
ONE OF THE BIG CLUSTER TOOLS. WE 
WILL TOUCH ON THAT LATER. JUST UPGRADE, 
POINT TO THE VERSION THAT YOU WANT 
TO -- YOUR DEPLOYMENT TO BE BASED 
ON AND FROM THEN ON, YOU SAW THAT 
CONTROLLER 
SERVICE THAT SITS INSIDE THE BIG 
DATA CLUSTERER. WILL TAKE OVER AND 
ROLL OUT THE SOFTWARE ACROSS ALL 
THE SERVICES THAT ARE IN BIG DATA 
CLUSTER. GOING TO DO IT IN AN OFFEFFICIENT 
WAY AND LOOK AT SERVICE DEPENDENCY. 
IT WILL LOOK AT CLUSTER HEALTH AND 
SERVICES HEALTH TO TAKE DECISION 
AROUND MOVING FORWARD WITH UPGRADE 
OR ROLLING BACK, IF IT HITS ANY 
ISSUES AND THINGS LIKE THAT. I TRIED 
ONE OF THE UPGRADES THE OTHER DAY AND IT WAS 
VERY, VERY COOL TO SEE. MY FAVOURITE 
WAS TO SEE THAT SEQUEL SERVER, IN 
THE AVAILABILITY GROUP CONCENTRATION. 
I HAD MULTIPLE REPLICAS OF PIT THERE 
IS NO MANUAL ORCHESTRATION REQUIRED, 
NO FAIL OVER TO ORCHESTRATE THAT. 
THAT WAS VERY NEAT. >> DOESN'T TAKE 
LONG. >> DOESN'T BECAUSE IT TAKES 
AS LONG AS DEPLOYMENT IF YOU THINK 
ABOUT IT. BECAUSE YOU HAVE TO PULL 
THE IMAGES ROLL OUT THE NEW SOFTWARE 
AND IT'S THERE. >> WE WILL HAVE 
TO SEE ALL THIS. KEEP GOING. >> 
THEN THIS CONTAINER BASED ARCHITECTURE, 
THAT IS SELF-CONTAINED AS I SAID, 
IS FAST, AND ENABLES THE FASTER 
WORK FLOWS AND PREDICTABLE BECAUSE 
YOU ALWAYS HAVING THE SAME THING, 
RIGHT. IT IS ALSO THE TYPE OF INFRASTRUCTURE. 
YOU MENTIONED THAT. DOESN'T -- IT 
DOESN'T MATTER WHAT TYPE OF KUBERNETES 
FLAVOUR YOU ARE RUNNING. YOU HAVE 
THIS FLEXIBILITY TO EITHER DEPLOY 
IT IN THE CLOUD OR YOU CAN DEPLOY 
IN YOUR OWN DATA CENTRE, YOUR OWN 
KUBERNETES LEGISLATURE. ALL -- SLURZ /* /* /* 
>> SET OF SERVICES THAT ARE DEPLOYED 
WITHIN THE CLUSTER WHEN YOU ROLL 
OUT -- STRAP THE PLATFORM. SO LET'S 
SEE WHAT ARE THOSE SERVICES. FIRST 
OF ALL, YOU TALK ABOUT DATA AND 
WANT IT TO BE SECURE. WANT THE DATA 
TO BE SECURE AND THE PLATFORM TO 
BE SECURE. YOU HAVE THIS AUTOMATIC 
SETUP ACTIVE DIRECTORY INTEGRATION 
AND AUTOMATICALLY CON FIGURE THE 
CONTAINERS TO ENABLE THE END-TO-END 
INTEGRATION WITH A. D. IT MEANS 
YOU USE YOUR ACTIVE DIRECTORY IDENTITY, 
CONNECT TO SEQUEL SERVER AND FROM 
THEN ON, IF YOU WANT TO PUT SOMETHING 
IN H. D. F. S. , SAME IDENTITY PASS 
THROUGH.  AND THE SAME IDENTITIES 
USED IN H. D. F. S. S.  >> THIS 
IS GOING TO BE STUNNING FOR YOU 
TO SEE. WE WILL GO THROUGH THAT 
ARCHITECTURE, WHERE YOU SEE THE 
WAY IT FLOWS DOWN THROUGH AND YOU 
ARE NOT HAVING TO CON FIGURE IT. 
IF YOU WANT TO CON FIGURE ACTIVE 
DIRECTOR, TO WORK AGAINST LINUX, 
IT IS QUITE THE CHALLENGE SOMETIMES. 
WE ARE HANDLING THAT FOR YOU. >> 
YES. AND THEN, THERE IS HIGH AVAILABILITY. 
WE ARE RUNNING IN KUBERNETES. KUBERNETES 
COMES WITH SOME GUARANTEES AROUND 
HIGH AVAILABILITY. BUT YOU HAVE 
THE OPTION TO DEPLOY WITH ADDITIONAL 
-- VERSES MISSION CRITICAL SERVICES 
LIKE THINK ABOUT SEQUEL SERVER, 
THINK ABOUT H. D. F. S. NAME NODE, 
SO YOU CAN DEPLOY WITH ADDITIONAL 
-- IT WILL ADD REDUNDANCY AND RELIABILITY 
FOR THE SERVICES. AND AGAIN, FAIRLY 
EASY WHEN THE DESIGN, THE LAYOUT 
OF THE CLUSTER, TELL NEWS THE CONFIGURATION 
FILE, I WANT MULTIPLE REPLICA OF 
THE COMPONENTS AND WHEN DEPLOYMENT 
HAPPENS WE AUTOMATICALLY DEPLOY 
CLUSTER ORCHESTRATER FOR YOU LIKE 
SCENE KEEP THEIR WILL CA -- LIKE 
ZOO KEEPER. GROUPS CONFIGURATION 
AND THE FAIL OVER LAPS ASPECT OF 
THAT FUTURE. >> WHAT WE ARE TALKING 
ABOUT HERE IS DESCRIBING IT IN A 
FILE AND SAYING THAT IS WHAT I WANT. 
AND THEN IT HAPPENS. IT IS A DECLARATIVE 
ENVIRONMENT, SO WE HAVE DECLARATIVE 
PROGRAMMING, DECLARATIVE COMPUTER 
AND DECLARATIVE SEQUEL SLUSER. >> 
WE WILL GET TO QUESTIONS IN A BIT. 
WE WERE SAYING WE WON'T HAVE TIME 
FOR A LOT OF QUESTION. THE QUESTION 
WAS, WHAT DO WE DO IF WE DON'T HAVE 
ACTIVE DIRECTORY AND DO HAVE A SECURITY 
MODEL FOR THAT? WE WILL COME BACK 
TO THAT AT THE VERY END.  >> SO 
DEFINITELY, THERE IS AN OPTION TO 
DEPLOY WITHOUT INTEGRATION. ANOTHER 
THING THAT I WANT TO TALK ABOUT. 
-- ABOUT, THE HEALTH SERGEANT AGENTS 
THAT ARE LOOKING FOR SIGNALS. ARE 
THOSE RESOURCES HEALTHY IN THEY 
ARE LOOKING FOR SIGNALS AROUND WHAT 
ARE THE STATE OF THE CLUSTERER? 
UP GRADING? DEPLOYING. ALL THE THINGS 
ARE COMPUTED AND THOSE VALUES ARE 
WRITTEN TO YOU, EITHER WHEN YOU'RE 
MONITORING THE CLUSTER LIKE I WANT 
TO SEE IF MY CLUSTER IS HEALTHY 
OR NOT. THEY ARE USED BY THE WORK 
FLOWS. WE MENTIONED EARLIER UPGRADE. 
THE CONTROLLER SERVICE, LISTEN TO 
THE SIGNALS AND TAKES THE DECISIONS 
ON ROLLING BACK, IF THE CLUSTER 
IS NOT HEALTHY. >> WE WILL SHOW 
THAT YOU PROCESS IN A MOMENT. >> 
THEN I THINK THIS IS THE FIRST SEQUEL 
SERVER THAT WE HAVE THIS BUILT IN 
MECHANISM FOR MONITORING AND TROUBLESHOOTING. 
SO WE HAD THIS MONITORING COMPONENT 
AND TAKING CARE OF COLLECTING METRICS 
FROM ALL THE SERVICES, COLLECTING 
METRICS FROM THE INFRASTRUCTURE 
LIKE FROM KUBERNETES NODES IN TERMS 
OF RESOURCES AND ALSO THE LOSS. 
FOR TROUBLESHOOTING PURPOSES WE 
COLLECT THE LOGS AND STORE THEM 
AND ALSO YOU HAVE BY DEFAULT, DASH 
BOARDS. PRECON FIGURED FOR YOU TO 
MONITOR THE VALUES OVER TIME OR 
TO SEARCH THROUGH THE LOGS FOR TROUBLESHOOTING 
PURPOSES. >> THAT IS IMPORTANT. 
WE HAVE A LOT OF COMPONENTS ROAMING 
AROUND. WE HAVE SEQUEL, GT SPARK, 
GOT H. D. F. S. , LINUX, ALL KIND 
OF THING. ALL OF THOSE HAVE DIFFERENT 
MONITORING AND LOGGING FACILITIES. 
SO WE DIDN'T WANT TO JUST SAY GOOD 
LUCK WITH THAT. AND MAKE YOU USE 
30 TOOLS TO DO THAT. TO LET YOU 
SEE AND LOG AND SEARCH THE LOGS 
ACROSS ALL OF THEM. SO YOU HAVE 
ONE PANE OF GLASS TO WORRY ABOUT. 
>> FIRST, I WANT TO TALK ABOUT WHAT 
ARE THE EXPERIENCES THAT WERE ENABLING 
WHAT -- AND WHAT ARE THE TOOLS THAT 
YOU HAVE AVAILABLE TO MANAGE ALL 
THESE THINGS? FIRST, YOU HAVE THE 
KUBERNETES CLUSTER ITSELF AND IF 
YOU DEPLOYED. IT, YOU MOST LIKELY 
USED -- TO DO MONITOR AND TO LOOK 
AT -- MANAGE THE CLUSTER. AGAIN, 
THE KUBERNETES CLUSTER. THIS IS 
A NEW PYTHON-BASED CROSS PLATFORM 
TOOL, TO DO EVERYTHING, IN CON FIGURING, 
TO DEPLOYING IT AND INTERACTING 
WITH SERVICES. WE ADDED EXTENSIONS 
TO BE ABLE TO SUBMIT SPARK JOBS 
TO RUN NOTEBOOKS TO TALK TO SEQUEL 
SERVER OR MANAGE FILES IN H. D. 
F. S. AND THINGS LIKE THAT. SO IT 
IS VERY GOOD FOR AUTOMATION. MANY 
OF THE CUSTOMERS THAT WE ARE WORKING 
WITH, THEY USE IT FOR SCRIPTING. 
LIKE YOU CAN SCRIPT ANYTHING FROM 
ALL THE CONFIGURATION, THE DEPLOYMENT, 
SCRIPT EVEN RUNNING WORKLOADS. YOU 
CAN RUN, WITH LOADS AGAIN. THE SERVICES 
THAT ARE IN THE CLUSTER. SO YOU 
CAN EASILIES YOU IT IN TEST AUTOMATION 
AND RELEASE -- BUT THEN FOR MORE 
GUIDED EXPERIENCE, YOU HAVE AZURE 
DATA STUDIO. IT IS AN INTERACTIVE 
EXPERIENCE. NOT ONLY THAT BUT IT 
IS INTEGRATED ACROSS BOTH THE MANAGEMENT 
ASPECT OF THE SERVICES, AND OF THE 
CLUSTER ITSELF BUT ALSO YOU CAN 
THINK ABOUT IT -- THIS IS THE TOOL 
TO INTERACT AND DO DATA NAIL  A 
-- ANALYSIS. YOU HAVE THE BUILT 
IN NOTEBOOK TO CREATE SPARK APPLICATION, 
TO SUBMIT SPARK JOBS, RIGHT? IN 
IS AVAILABLE FOR YOU. YOUR ANALYSTS 
WILL WORK TOGETHER TO COLLABORATE 
IN WRITING THIS -- >> HOW MANY OF 
YOU FAMILIAR WITH JUPITER NOTEBOOKS 
IN DO YOU SEE ANYTHING WEIRD UP 
THERE? >> POWER SHELL. >> POWER 
SHELL AND SEQUEL COL COLONEL. WE 
HAVE A SEQUEL KERNEL AND SPARK KERNEL 
AND PYTHON KERNEL AND SO ON. INSIDE 
THIS TOOL. >> YEAH.
SO ALSO FROM 
MANAGEMENT PERSPECTIVIVE, BECAUSE IN THE 
SESSION WE 
ARE TALKING ABOUT MANAGING THE CLIRSS, 
CLUSTERS, WE HAVE DEPLOYMENT. THE 
RELEASE OF AZURE STUDIO, HAS DEPLOYMENT 
WIZARD THAT YOU CAN USE -- THAT 
WILL GUIDE YOU THROUGH THE STEPS 
FOR DEPLOYING THE BIG DATA CLUSTER. 
MAKE SURE YOU USE THAT AND WE WILL 
SEE SET OF BUILT IN NOTEBOOKS. SAW 
EARLIER NOTE BOOKS. WE WROTE NOTEBOOKS 
FOR YOU AND ARE SHIPPING THEM WITH 
THE AZURE DATA STUDIO TO GUIDE THROUGH 
TROUBLESHOOTING REPAIRING ITEMS 
OR DIAGNOSING DIFFERENT ISSUES YOU 
MAY ENCOUNTER. SO THAT'S PRETTY 
COOL AS WELL. YOU CAN ALSO USE AZURE 
STUDIO TO MONITOR. SO WE HAVE SERIES 
OF DASH BOARDS THAT YOU CAN LEVERAGE 
TO LOOK AT THE CLUSTER STATUS IN 
GENERAL. YOU CAN USE IT TO FIND 
DIFFERENT END POINTS THAT YOU CAN 
CONNECT, BECAUSE AGAIN, WE HAVE 
SEQUEL SERVER, HAVE ALL THE WEB-BASED 
DASH BOARDS, HAVE SPARK, RIGHT? 
SO THE DASH EBOARD, AZURE STUDIO, 
THEY WILL HELP YOU TO EASILY FIND 
THE END POINT. >> BIG QUESTION, 
HERE IS, WHEN I JOIN SEQUEL SERVER 
BACK IN 2006, THE TOOL I WROTE WAS 
MANAGEMENT STUDIO AND SO THE MOST 
IMPORTANT THING IS CAN I STILL USE 
MANAGEMENT STUDIO? >> WHAT DO ALL 
YOUR SEQUEL INTERACTION, CONTINUE 
USING SEQUEL SERVICE MANAGEMENT 
STUDIO BUT BIG DATA CLUSTER SPECIFIC 
THINGS LIKE NOTEBOOK AND SPARK, 
IS AZURE STUDIO. >> LET ME ASK ONE 
QUESTION. ARE THERE ANY MANAGE, 
PEOPLE WHO MANAGE FOLKS IN THE ROOM? 
SORRY, I WILL TALK A LOT SLOWER. 
[ LAUGHTER ] WILL THIS LIKE THE 
-- THERE THAT EXPORT TO EXCEL? >> 
WHAT DO YOU THINK?  >> AND THE ANSWER 
IS, IT WILL. IF YOU ARE A MANAGER 
YOU SHOULD FEEL VERY CALM RIGHT 
NOW. YOU CAN SEND THAT IN AN E-MAIL. 
MAYBE DO A POWER POINT WITH A PIE 
CHART, IT WILL BE GREAT. [ LAUGHTER 
] ANYWAY. >> AGAIN. YOUR FAVOURITE 
FCABANA AND GRAFANA. THE DASHBOARD 
-- YOUR THIS IS JUST A LOT. -- >> 
AFTER WE TAKE A LOOK WE WILL JUMP 
RIGHT INTO THE TOOLS A TAKE A LOOK. 
>> EXACTLY. WE TALKED ABOUT TOOLS. 
TALKED ABOUT THE BENEFIT OF THE 
INFRASTRUCTURE AND SOME OF THE SERVICES 
THAT WE ARE ENABLING. NOW WE WILL 
LOOK INSIDE THE BIG DATA CLUSTER. 
BUILT IN POLL KOENLTS THAT -- COMPONENTS 
THAT, WITH TOGETHER. >> SO WE HAVE 
OUR CLUSTERER. >> HAVE THE CLUSTERER. 
KUBERNETES. IT HAS ITS OWN MASTER 
SERVICE WE TALKED ABOUT EARLIER. 
>> WE JUST MANAGED THAT WITH CUBE 
CUDDLE. THE NORMAL COMMAND. IT WILL 
BE EXACTLY THE SAME. >> THE FIRST 
THING YOU SAID IT EARLIER, THE FIRST 
COMPONENT THAT COMES UP WHEN YOU 
BOOT STRAP THE CLUSTER. EASY DATA, 
THROUGH THE AZURE STUDIO, DEPLOYMENT 
WIZARD AND FIRST THING THAT COMES 
UP WITH THE CONTROLLER SERVICE. 
>> INTERACT WITH THAT. >> THIS IS 
EASY DATA. AZURE DATA STUDIO, ENABLED 
A LOT EVER EXPERIENCE TO MANAGE 
THE CLUSTER THROUGH THE CONTROLLER. 
THAT'S BOTH OF ARE AVAILABLE TO 
USE. ALL THE DATA THAT SUPPORTS 
THE MANAGE EXPERIENCES IS STORED 
AND GUESS WHAT THAT IS? >> WHAT 
IS THIS POD? >> SEQUEL SERVER. >> 
ANOTHER SEQUEL SERVER. >> STORING 
ALL THAT DATA. >> GOT IT. >> DEPLOY 
THAT SERVICES, WE TALK ABOUT EARLIER, 
ABOUT UPGRADE, CONTROLLER WATCHDOG. 
UPGRADE ORCHESTRATION. THEN THE 
OPERATOR, WE ARE GOING MENTION IT 
I THINK IN THE NEXT SLIDE.  IS INVOLVED 
IN THE ORCHESTRATION FOR HIGHER 
AVAILABILITY, SER SECRET SERVER 
MASTER AND HAVE THE MONITORING AND 
TROUBLESHOOTING COMPONENTS. RIGHT? 
SO THIS IS -- THE WAY WE INTERACT 
IS THROUGH THIS INTERFACE, THAT 
YOU CAN ACCESS THE DASH BOARDS THAT 
ARE SITTING BEHIND THIS PROXY WE 
HAVE AGAIN DEPLOYED BY THE -- >> 
SO I TALK TO THE MANAGEMENT PROXY. 
IT TALKS TO GRAFANA. IT IS SIMILAR 
TO THE PERFMAN IN WINDOWS. THE EXEMPT 
IS YOU CAN CON FIGURE IT. >> YOU 
CAN ADD YOUR OWN, ADD YOUR OWN DASH 
BOARDS, KUS ONLY MIZE IT. -- COME 
TOM MIZE IT. >> HOW DOES GRAFANA 
READ ALL THE STUFF HAPPENING IN 
>> GRAFANA TAKES TO THE DATA IT 
EXPOSES. THEN HOW THIS METRICS GET 
THERE IS THROUGH THIS SERVICES DEPLOYING 
IN THE CLUSTER. TELEGRAPH IS PICKING 
UP THE MET TRICKS FROM THE HOST, 
FROM KUBERNETES NODES. >> THESE 
ARE DEPLOYED ON EACH OF THE NODES 
THAT PARTS. >>> THEY ARE COLLECTING 
DATA FROM THAT NODE. PUTTING UP 
IN INFLUX? >> THEY WILL STORE THEM 
IN INFLUX. >> THEN I READ INFLUX 
THROUGH GARFANA. BECAUSE IT IS TIME 
SERIES, INTERLEAVED SO I CAN SEE 
A SE SEQUEL SERVER WAS HAVING A 
BAD DAY. I SEE THE TWO THINGS TOGETHER. 
>> YOU CAN SEE.  >> GOOD STUFF. 
ABOUT WHAT LOGGING THOUGH IN >> 
THE LOGGING SO THE WAY YOU SEE THE 
LOGS AND CAN SEARCH THROUGH THEM 
IS THROUGH CABANA. RIGHT? THE DASHBOARD 
AS WELL. SO YOU CAN CON FIGURE YOUR 
OWN CH QUERY AND SAVE THEM. WHERE 
THE LOGS ARE -- AND DEPLOYED BY 
DEFAULT AND IN THE CLUSTER, IN ALL 
THE PODS, SO IF YOU LOOK AT THE 
POD IN ONE OF THE POD. PDOESN'T 
MATTER, YOU HAVE TO -- IT TAKES 
OUT THE LOGS THEY ARE INDEX, SO 
YOU CAN EFFICIENTLY E CH QUERY. 
>> GRANANA CABANA IS HOW I DO THIS. 
I CAN GET TO THAT WITH EXCEPT THROUGH 
THE CUBE CUD.  IT IS LAUNCHING POINT? 
>> WE. I WILL SHOW YOU HOW WE HAVE 
EASY TO LAUNCH POINT. PROVIDING 
YOU THE BUILT IN QUERY FOR A DIFFERENT 
SPECIFIC. >> WE WILL HAVE HER DO 
THIS. BUT LET'S TALK ABOUT THE HIGH 
AVAILABILITY PART. CAN YOU TELL 
ME MORE AND THAT. >> WE ARE DEPLOYING 
IN KUBERNETES. WE ARE USING -- KUBERNETES 
IS RESPONSIBLE TO ENSURE HEALTH 
OF THE SERVICES. IF A POD FAILS, 
IT WILL RESTART BY ITSELF. >> THAT 
IS JUST PART OF KUBERNETES. >> CAN 
DEMRI PLOY SOME SERVICES. YOU CAN 
DO THE SAME THING FOR H. D. F. S. 
NAME NODE THROUGH ZOO KEEPER. FOR 
SEQUEL SERVICE MASTER, WHEN YOU 
WANT TO DEPLOY IT WITH MULTIPLE 
REPLICAS WE WILL CON FIG AN AVAILABILITY 
GROUP FOR YOU BUT BEFORE THAT, WE 
ARE DEPLOYING THIS HOP RATER. TALKS 
TO THE CONTROLLER. TO NOT ONLY DEPLOY 
LET'S SAY, YOU SPECIFY THREE REPLICAS 
FOR THE INSTANCE, WE ARE NOT JUST 
DEPLOYING THREE REPLICAS OF THREE, 
OF SEQUEL SERVER BUT ALSO CON FIGURING 
-- YOU DON'T NEED TO CON FIG MANUALLY 
END POINT. YOU DON'T NEED TO JOIN 
YOUR REPLICAS TO THE AVAILABILITY 
GROUP. MORE THAN THAT, WHEN YOU 
CREATE A DATA BASIC IT WILL BE AUTOMATICALLY 
ADD TO THE AVAILABILITY GROUP. IF 
YOU OPT INTO THE FEATURE YOU GET 
EVERYTHING. >> TELL ME ABOUT THE 
FAIL OVER. SHOW NE A FAIL OVER. 
>> FIRST, WE HAVE AN APPLICATIONS 
CONNECTED TO AN END POINT THAT YOU 
AGAIN, YOU HAVE IT CON FIGURED. 
THIS IS YOUR AVAILABILITY GROUP 
LISTENER. IF YOU FAMILIAR WITH THE 
FEATURE THAT, WHAT THE END POINT 
DOES. IT WILL ALWAYS POINT TO THE 
PRIMARY. IF IT HAS A FAILURE, RIGHT, 
IT WILL POINT TO THE NEW ELECTED 
PRIMARY AUTOMATICALLY. SO AGAIN, 
THIS IS EXACTLY WHAT AVAILABILITY 
GROUP LISTENER DU. >> SURE, SURE. 
>> SECOND END POINT IS DEPLOYED 
FOR YOU FOR READ ONLY APPLICATIONS. 
SO THIS IS DOING EXACTLY THE SAME, 
EQUIVALENT FUTURE, WITH THE SCALE 
OUT WE HAVE, AND IN REGULAR -- >> 
REGULAR SEQUEL SERVER. >> AND THE 
ONLY CONFIGURATIONS IS THERE FOR 
YOU. YOU DON'T HAVE TO CON FIG THE 
REPLICA. >> WE HAVE GOT MANAGEMENT 
AND MONITORING. TALKED ABOUT HIGH 
AVAILABILITY. TALK ABOUT SECURITY 
A BIT. BECAUSE THERE IS LOTS OF 
THINGS TO SEE CURE. THE WAY WE DO 
THIS -- -- TO SECURE THIS. LET'S 
TALK ABOUT HOW THAT SECURITY WORKS. 
SO WE TALK A MOMENTS AGO ABOUT MANAGING 
AND MONITORING THE CLUSTER. THE 
WAY YOU DO THAT IN KUBERNETES IS 
ISSUE A CUBE KETTLE GET CREDENTIAL 
COMMAND AND BRING DOWN A CERTIFICATE 
THAT ALLOWS YOUR SYSTEM THAT YOU 
JUST DID THAT ON, TO TALK TO THE 
CLUSTER. THAT IS SECURE COMMUNICATION 
CHANNEL, MENTALLY YOU CAN THINK 
OF THIS LIKE O. D. B. S. S.  YOU 
HAVE A FILE, HIDDEN ENCRYPTED AND 
THAT TALKS BACK AND FORTH. THAT 
DOES NOT CHANGE. WE DON'T TOUCH 
THAT. THAT IS YOUR KUBERNETES, DEPENDING 
ON HOW YOU DO THAT. HOW DO WE GET 
THIS INTEGRATION WITH ACTIVE DIRECTORY 
IN YOU AWE THEN KATE WITH ACTIVE 
DIRECTORY. THOSE OF YOU WHO KNOW 
ACTIVE DICK DIRECTORY, DO YOU WANT 
A JOB? WE HAVE A LOT OF OPENING, 
NO, AC ACTIVE DIRECT TOIR IS REALLY 
PERRY BELLGARDE. THE TOKEN IS SEN 
OFF TO THE CONTROLLER INSIDE THAT 
WE TALKED ABOUT EARLIER AND NOW 
IT HAS THE TOKEN SO IT CAN DO CONTROLLER 
BASED OPERATION. SEQUEL SERVICE, 
THE WAY THAT H. D. F. SPCHLT WORKS 
IS SOMETHING CALLED ACCESS CONTROL 
LISTS, THAT IS STANDARD UNITS PRACTICE. 
SO WE CREATE A SERVER MASTER KEY. 
AND A DATA BASE MASTER KEY. AUTHENTICATE, 
DOWN H. D. F. S. , ASSIGN THE, A 
KLS. WHAT IF WANT TO TALK TO E. 
C. 2 OR TALK OUT TO A. D. L. V. 
V3, WANT TO TALK TO S3? THE WAY 
WE DO THAT, LET ME GO BACK. IF I 
WANTED TO TALK TO SPARK, I SI SIMPLY 
TALKS TO KNOX. KNOXS WILL CREATE 
A TOKEN, GOES THROUGH TO LIVY CHEQUE 
IS JOB SCHEDULING SYSTEM. THERE 
IS THE YARN WE SAW EARLIER AND PASSED 
DOWN. I JUST LOGGED IN ONE TIME, 
THEN IF I DO IN FACT WANT TO TALK 
TO A TIERING PROCESS HERE, LET ME 
GO BACK HERE. BASICALLY, IT GOES 
DISH WILL USE THE CONTROLLER -- 
TO ONCE AGAIN, IT IS REGULAR OLD 
ACCLES. I WANT TO SEE PIT IN TWO 
MINUTES AND 38 SECOND, LET'S TAKE 
A LOOK AT WHAT YOU HAVE GOT AVAILABLE 
TO US HERE. SO WHAT AM I DOING HERE? 
>> AGAIN, WE TALKED ABOUT ACTIVE 
DIRECTORY INTEGRATION. I CONNECTED 
TO THE SEQUEL SERVER. RECOGNIZES 
IT IS A PART OF A BIG DATA CLUSTER 
AND ENABLES EXPERIENCES SPECIFIC 
TO. THAT CREATE NOTE BOCK, OPEN 
A CLUSTER DASHBOARD, THAT GIVES 
ME THE END POINTS TO OTHER SERVICES. 
THIS IS SEQUEL SERVICE MASTER IN 
AVAILABILITY GROUP CONFIGURATION 
BECAUSE I HAVE THIS SECONDARY END 
POINT AND ALSO MONITOR MY CLUSTER 
BECAUSE I CAN SEE SERVICE PATTERNS 
SO FOR EXAMPLE, FOR SEQUEL SERVER 
HERE ARE ALL THE LAYERS. >> THERE 
THEY ARE. >> TALK ABOUT -- >> THESE 
ARE THE NOTES WE TALKED ABOUT.  
>> THEY ALL HAVE SEQUEL SERVER AND 
I CAN JUMP TO GRAFANA TO CABANA. 
>> SHOW ME THAT. I WANT TO SEE THAT. 
>> WE HAVE THE SEQUEL METRIC, THE 
NODE METRICS. EXCELLENT COLLECTING 
DATA -- COLLECTING DATA AS WELL 
AND COLLECT COLLECTING LOGS. THIS 
IS THE DASHBOARD AND PANELS. I'M 
NOT KIDDING, IT DOES EXPORT TO EXCEL. 
YOU CAN MAKE YOUR OWN. >> YOU CAN 
MAKE. WE HAVE THE ADD PANEL. YOU 
CAN CREATE. >> WE BUILT THESE FOR 
YOU. ONTARIO WE HAVE FOR -- >> WE 
HAVE FOR THE HOST, AS -- WE HAVE 
DASHBOARD FOR YOU TO USE FOR SPARK. 
BUT YOU CAN CREATE YOUR OWN AS WELL. 
THEN KA BAN NARC EASY TO NAVIGATE 
THROUGH THE LOGS HERE. YOU CAN BUILD 
YOUR OWN QUERIES. THIS IS AT DASHBOARD 
THAT SHOWS THE LOGS. ONE OF THE 
POD.  I CAN FILTER IT DOWN TO JUST 
SHOW ME SEQUEL SERVER. SEQUEL SERVER 
LOGS. SORRY. I DON'T KNOW. >>> THIS 
-- I MEAN THIS KIND OF STUFF IS 
NEW. TO YOU PROBABLY, HAVING A SINGLE 
VIEW OF ALL THE THINGS. THIS WAS 
NO SMALL FEAT TO GET IT INTO THIS 
VIEW. BUT YOU -- WE ANTICIPATE -- 
>> IT FILTERED FOR SEQUEL SERVER 
TO GIVE ME JUST THINK ABOUT THIS. 
THIS IS ERROR LOG.  AND I CAN EASILY 
SEARCH LIKE FOR EXAMPLE, IF I -- 
I HAVE ERROR. >> SO SEARCHES THROUGH. 
I CAN SEE THE LOGS. YOU CAN SEE 
THEM HIGHLIGHTED HERE AND THIS IS 
ONCE AGAIN, INTERWEAVED. SO I CAN 
SEE THE ENTRIES. I CAN NOT STRESS 
HOW HARD THAT IS TO DO WHEN YOU 
DON'T USE THESE TOOLS. SO IF YOU'RE 
A DATA PROFESSIONAL OR PERSON, AN 
ARCHITECT, LEARN KUBERNETES, LEARN 
CONTAINER AND GRANANAN AND CABANA. 
WE HAVE A LOT OF RESOURCES. SO I 
WILL GO AHEAD AND POP BACK OVER 
TO SLIDE WORLD IN THE INTEREST OF 
TIME. BECAUSE WHAT I DO WANT TO 
DO -- JUST SHOWING THE SPARK LOGS 
SO YOU HAVE THE TYPICAL UIS. YOU 
CAN SEE THE JOBS GET THEIR I. D. 
AND SEE WHAT THE JOB IS DOING. IF 
YOU WANT TO WRITE JAVA, TAPE KEYBOARD 
TO YOUR FEET. YOU CAN TELL I LIKE 
JAVA A LOT. HERE IS THE UPSHOT HERE. 
WE HAVE THE GOT THE OFFICIAL DOCUMENTATION 
RIGHT HERE. WE HAVE GOT IN-DEPTH 
TRAINING. SOME OF THE OTHER THINGS, 
ALL OF THE STUFF YOU GET IN SEQUEL 
SERVER 2019 IS IN HERE BECAUSE THIS 
IS 2019. JUST RUNNING THERE AND 
THE TELL -- TEAM PUT TOGETHER EXCELLENT 
BIG DATA CLUSTER SAMPLE. SO YOU 
HAVE GOT THOSE AVAILABLE TO YOU 
IN NOTEBOOK FORMAT. ANOTHER THING 
FOR YOU TO PUT DOWN, LEARN JUPITER 
NOTEBOOKS. I THINK THIS HAS BEEN 
LIKE GOING THROUGH A CAR WASH SO 
FAST, RIGHT? I NO I KNOW IT IS A 
LOT -- I KNOW IT IS A LOT. WE DO 
HAVE SLIDE. THERE IS OTHER SECTIONS 
AVAILABLE.  I WILL CLOSE OUT HERE. 
I WILL BAIL OUT ON YOU. AND HEAD 
SOMEWHERE ELSE. I REALLY APPRECIATE 
YOUR TIME THIS MORNING. WAS THIS 
USEFUL, FOLK S?  [ APPLAUSE ] >> 
THANK YOU.  >> THANK YOU. AND WE 
WILL SEE YOU AT THE NEXT SESSION. 
