RETO MEIER: Google
Cloud Next 2017
hosted a veritable
smorgasbord of talks
detailing the future of cloud.
If you didn't get a chance
to catch BigQuery and Cloud
Machine Learning, Advancing
Large Scale Neural Network
Predictions, then stay tuned,
because here is the recap.
[MUSIC PLAYING]
One of the common challenges
with using machine learning
is storing, processing,
and supplying
huge volumes of training
data to your ML Engine.
COLT MCANLIS: Thankfully,
Google Cloud Platform
makes this super easy with
the combination of BigQuery,
a petabyte scale enterprise data
warehouse, and Cloud ML Engine,
which is a fully managed
TensorFlow platform.
Used together, BigQuery
can supply huge volumes
of training data
to ML Engine, which
can create training
and prediction
models, which can be distributed
and used on various platforms.
RETO MEIER: The
most common use case
for combining BigQuery
and ML Engine is
to use BigQuery as a data lake.
You can store all the data
you're interested in, even
if you're not using it all, and
preprocess various parts of it
before exporting it to
TensorFlow or ML Engine.
You can then use
your trained models
to enable more advanced
queries of your company data.
COLT MCANLIS: Now
it's really common
to run searches on your data
using keyword matching, which
is specifying tags you
can use to find content,
such as the actor's name, the
movie name, or the music type--
anything you can find in the
document or its metadata.
While useful, tag-based
searches are not
as powerful as using
feature vectors, which
are a list of tuples containing
words and their "importance
score," relative to the rest
of the words in the document.
And since BigQuery lets you
create user-defined functions
within SQL queries, we can
write JavaScript functions that
calculate the feature
vector for each document
and determines how similar
they are to each other.
You could use this technique
to find similar questions
on Stack Overflow or
similar posts on Reddit.
RETO MEIER: You can use the same
approach for all content, not
just documents.
And that's where ML
Engine comes into play.
We can create ML models
to extract feature vectors
from unstructured data, like
images, natural language,
or spending patterns, and
then use BigQuery's UDFs
to do feature vector
queries using those models.
When applied to images, instead
of getting a list of labels
describing the contents
of each picture,
we get feature vectors
that can be compared
to those from other images.
As a result, even without
knowing what's in each image,
we can search BigQuery
for similar images
or perform analysis based on
grouping those similar images.
COLT MCANLIS: Another
use case for combining
BigQuery and machine learning
is to perform large-scale demand
forecasting and
providing recommendations
for your customers.
We're once again going to
take advantage of BigQuery's
user-defined functions.
This time, we'll use
historical buying trends
to train models
using the ML Engine.
Those trends can be
based on variables,
including the month, the
season, or even the weather.
RETO MEIER: Or you can
use the buying trends
of groups of users to
predict what products you
should be recommending them.
Once you've
developed the models,
you can store them
in Cloud Storage
and make forecasts
and predictions
for specified
conditions being passed
as inputs to user-defined
functions within BigQuery.
To understand more
about feature vectors
and see a bunch of examples
of how to use ML Engine
and BigQuery together, check
out Kaz's full session video
for all the details.
And if you want more recaps
on great Next content,
make sure to check out
the rest of our playlist.
And don't forget
the next world tour,
coming soon to a city near you.
[MUSIC PLAYING]
