[MUSIC PLAYING]
JOHN PALLETT: I am part
of the Chrome media team.
Our job is to make
video and audio
on the web just fantastic.
And we want as many
sites delivering media
to as many users as possible.
To that end, there's
been a lot of new web
APIs over the last few
years, especially lately
from mobile media playback.
And it's really
exciting for us when
we see sites using these APIs.
For example, Forbes launched
a progressive web app.
And when developing
their narrative
for the progressive
web app, they
made video an integral
part of the experience.
iHeartRadio launched a
web streaming experience.
And using media session
API and service workers,
they were able to give
their users the experience
that they expected.
And then, at the
beginning of this month,
Geo Cinema launched a
PWA for media playback.
It's only been a few
weeks, but they're already
seeing 10% more session time
in their PWA, on average,
than in their native app.
So there's a lot of
activity going on
in web media, and especially
mobile web media right now.
And we've seen a growth in
mobile web media watch time.
In fact, per user
media session time
has increased by double digits
over the last six months
in Chrome on Android.
So what are we
going to talk about?
This is the last
talk of the day.
And we're going to do
it in three sections.
The first thing I'm going to
do is give you a few updates
from the Chrome media team.
And next, Francois
is going to join us.
And he's going to talk about
some practical applications
of media and how
to get things done.
And finally, the team behind
the voot.com progressive web app
will talk about
their PWA experience.
And they're also going to talk
about some new work they're
doing for offline
media playback.
The bus icon will make
a little more sense
once they start talking
about what they're doing.
But it's really
interesting, because it even
is providing media for users who
have no internet access at all.
So stick around for that part.
It's a really neat
application of the web.
So let's start with
a couple of updates.
First, let's talk
about AutoPlay.
It's probably not a
big secret for anybody
in the room that one of the most
common complaints with Chrome
is when audio starts playing,
and the user didn't want it,
or they didn't
expect it, and it is
just-- it's a common complaint.
And frankly, it's
one of the reasons
that users install ad blockers.
And it's also been
difficult for developers,
because the policies haven't
always been consistent
between desktop and mobile.
Now, the fact is that AutoPlay
has a lot of great applications
when users are expecting it.
And so the team has been working
to try to unify everything
and make it easier
for everybody.
So you're looking at
what's best for the user,
and what's best for developers.
The primary goal was
to unify everything
across desktop and mobile.
And so as we talk
through these policies,
these apply to both
desktop and mobile
to make everything as
predictable as possible.
On all platforms,
muted media will always
be allowed, including
media in iframes.
You can always
AutoPlay muted media.
Unmuted media will be
allowed in three situations.
First, if the user has already
interacted with your site.
And what we mean by that
is a tap or a click.
And when we say
that, what we mean
is that if they go to one
page and they tap or click,
and then they go to
another page on that page,
media can play with sound.
Second, on desktop, if
the user has interacted
with the site a lot
with audio, we're
going to assume that audio
is allowed for the site.
And finally, on
mobile, if the site
has been added to
the home screen,
that's a pretty good indication
that the user doesn't
mind AutoPlay with sound.
In terms of code,
and in terms of how
you detect whether or
not AutoPlay is allowed,
you can do this by checking
the play promise after you
try to play media with audio.
If the promise fails,
if it's rejected,
then you know that only
muted AutoPlay is allowed.
One of the things that's
important to know as well
is that iframes always
can do muted AutoPlay.
But this site can
grant permission
to access the site policy.
And that's going to be
done using an attribute.
The attribute name
is not yet finalized.
But it's going to be
something like gesture equals
media or something like that.
And so two things to know about
AutoPlay before we move on.
Number one, what is
the best approach?
What we recommend, based
on our conversations
with users and the work
that we have been doing,
is muted AutoPlay.
Allow users to self-select
into the audio experience.
Now, that's not going
to work for every site.
In some cases, click to
play may be more applicable.
But that is our
recommendation in general.
The other thing that
we recommend doing is--
because there's a lot of
technical details to this--
keep an eye on the
AutoPlay policy page,
the bottom most link there.
That's where you'll see updates
as the dates and the details
around things like
the attributes change.
Next, let's talk
about Shaka Player.
For those of you who aren't
familiar with Shaka Player,
it's an open source
free player that
allows you to do adaptive bit
rate streaming fairly easily.
It gives you smooth
playback for video.
Instagram is going to go into
a little more detail about how
they use Shaka
tomorrow in their talk.
But for now, what I'm
going to talk about
is things that are new
in the latest release.
Shaka 2.2.4 was
actually just released
within the last 48 hours.
And it includes all
of the features here.
It includes support for
Apple HLS for on-demand.
That's with fragmented mp4.
It also allows sites to
customize the rendering
of web BTT and TTML.
And finally, it allows offline
protected media playback.
That's today something you can
try in Chrome on Android 62
and later.
Coming soon will be Apple
HLS support for live video,
including support for MPEG
to transport stream files.
That's a fairly
common HLS format.
They're also working on adding
background fetch for offline.
And that's really
going to round out
the offline story by allowing
users easier download flows.
And finally, Shaka
has a demo player.
And the player today
includes examples
on how to do things
like full screen
and rotate, which
Francois will cover,
as well as they're working
on adding progressive web app
features to the demo to
give you reference code.
So those are the updates.
So now let's switch over to
what sites are doing today.
The web is really
great for media.
You can write a great
media experience.
Once it works on web,
it works on mobile.
And typically, what you want
is you want fast playback,
you want a great
UI, and you want
people to be able to access it
anywhere, including offline.
Not everybody knows how to do
that, especially on mobile.
So what I'm going to do is
hand it over to Francois.
He's spent the last
six months working
on articles and examples on
how to give a great media
experience on mobile.
FRANCOIS BEAUFORT: Hello.
So here's a fact.
After two seconds
of buffering, user
starts dropping off at
around 6% per second.
In other words,
faster playback starts
means more people
watching your videos.
So let me walk you through two
techniques you can use today
on the web to expedite your
media playback on first load
by actively
pre-loading resources.
First, if your video
service is a unique file
hosted on a web server, you can
use the video preload attribute
to provide a hint to the browser
as to how much information
or content to preload.
Resource fetching will stop only
when the initial HTML document
has been completely
loaded and parsed,
while the Window load event
will be fired when resource
has actually been fetched.
Setting the preload
attribute to auto
indicates that the browser
may cache in-app data,
that complete playback is
possible without requiring
a stop for further buffering.
There are some caveats, though.
As this is just a
hint, the browser
may completely ignore
the preload attribute.
When user has enabled
Data Saver, for instance,
Chrome won't preload anything.
And on a standard
connection, Chrome
will only fetch media
metadata, such as dimension,
tracklist, duration, and so on.
So here's the second technique,
my personal favorite.
Link preload.
Link preload is a
declarative fetch
that allows you to
force the browser
to make a request for a
resource while the page is still
downloading without blocking
the window load direct.
Resource loaded
with link preload
are stored locally
in the browser
and are effectively inert until
they are explicitly referenced
in the DOM JavaScript or CSS.
So this code shows how
to preload a full video
on your website so that
when your JavaScript asks
to fetch video content,
it is read from cache.
As a resource may have already
been cached by the browser,
if the preload regress
hasn't finished yet,
a regular network
fetch will happen.
Note that I'm using as video
here in the link preload,
as I'm going to use that
resource as a video.
If it were an audio element,
I would be using as audio.
So what about preloading a
chunk of a video, you may ask?
In that case, I
would use ask fetch.
This is how to preload the
first segment of the video
with link preload and
use it with media source
extensions also known as MSC.
If you are not familiar
with MSC, that's OK.
Let's simply say
that this API is
what makes adaptive media
streaming possible today
on the web.
Netflix, Twitch, and
YouTube use it extensively.
For the sake of simplicity,
the entire video
has been speaking to Spotify,
like file 1, file 2, file
3, et cetera.
The goal is to feed all
these junk files to a video
so that playback is smooth.
So here's what happens.
When the media source
is created and ready,
I'm fetching the first segment
of a video that may already be
in the browser cache-- by the
way, thanks to link preload--
and upending that data to a
buffer belonging to the media
source of a video element.
And that is pretty much it.
Last thing.
As link preload is not
supported everywhere yet,
you should detect
its availability
with this [INAUDIBLE] in order
to adjust your preference
metrics.
Now, it's not
because you know how
to preload content to extract
media playback that you should
do always do it no matter what.
There are many, many
signals in the web platform
you can use to provide
a delightful media
experience to all users,
including the one we've limited
our media network connections.
Include the device battery,
memory, network connection
type, whether or not a video has
a poster image, the available
storage left, and whether or
not the data saver is enabled.
I have created a
tiny demo page that
takes a message of all
the signals at this URL.
And if you're trying
it now, you'll
see that the video may not
preload for all of you.
And that's because we
are all different--
and beautiful, by the way.
So let's start with
the first signals--
the device battery.
As usual now, this is
a progress enhancement.
If the battery is too
low and not charging,
let's not assume blindly that
preloading a [INAUDIBLE] video
will make user happy.
We could either lower
the quality of the video,
or even better in my
opinion, not preload at all.
If user is on a
low-end device, you
may want to be kind there
and not preload any media.
Just wait for user
to ask for it,
as the device may not be able
to handle everything smoothly.
I assume user is
on low-end device
if its device memory
is less than one gig.
And as you can see, it is pretty
easy to get that information.
Note that a device memory
HTTP header is also
available in Chrome.
That means if your
web server knows
how to handle these HTTP header,
also known as client hint
hitter, you can serve
an appropriate response
based on the device memory.
Estimating how much available
space is left on the device
is crucial.
And if you know there's
not enough to preload also
your video, you should
stop right away.
You may also use that
information to gray out a make
available offline burden.
That is why I recommend you
always provide a way for users
to clean oldest of
data on your web site.
Now, if user is on a
cellular network connection,
you should assume the network
data block is not infinite,
and they may even
have to pay for it.
So please be mindful.
Use the network
information web API
to detect user network
connection type
and act accordingly.
Hopefully, a dedicated
media property
will be added to
disappear in the future
to make this even more reliable.
And finally, here's
one way to detect
in JavaScript if
user has installed
the data saver in Chrome.
All you need is to
create a video demand,
set the preload
attribute to auto,
and check that the
reflected value is at none.
I know, it could be better.
And it will.
A save data boolean
attribute is in the process
of being added to the
network information API.
So fingers crossed.
By the way, there is also a
save data HTTP header also
available in Chrome.
So you might want to check
this if your web support--
if your web server
supports it, once again,
and serve an
appropriate response.
Now, a great media
user experience
relies on many
progress features.
And I'm here today to tell you
that the web platform is ready.
And I'm really excited.
Like, I actually mean it.
Let's start simple.
There are a lot of cases
where user may simply
want to listen to audio.
And if that is a primary
use case for you,
you should definitely use
the media station web API.
This API allows you to
put some custom media
metadata in an image
in the notification
accessible from the lock screen,
as well as on wearable devices.
It is also nice in
general because users
can tell what's going
on on their device
and easy to control
media playback.
One cool thing I've
noticed recently
is that it also works with the
Google Assistant on Android.
User can simply say, OK, Google.
Post music, or fast forward.
And it just works.
So let's have a look at how to
use the media session web API,
if you will.
As I said before, this
is a progressive feature.
So it starts with
a if statement.
If the browser supports it,
you can provide some media
metadata, like the
title, artist, album,
and a list of artworks.
You can specify many
more artwork sizes
than I did in this snippet.
The browser will always
pick the one that
is best for the user's device.
I would suggest, though, you
always provide 256 and 512
pixel squared images,
as they are the most
common ones on Android.
Once you've provided
some metadata,
you may also want to
respond to the media
controls exposed in the
notifications, such as seek
backward and forward,
play, pause, next track,
and previous track.
And for that, all you
have to do is set up
some action handles so that
when this action happens
you can take care of them.
Now, if you have some custom
media controls in your web
page, you can make sure
the state of your UI
is reflected in the notification
by overriding the playback
state to playing opposed.
I personally really
love this API
because it is simple and
powerful at the same time.
Let me show you, for
instance, some custom action
handles for seek
backward and forward.
This code is pretty simple
to understand, right?
These one liner
functions are only
about countering the current
and property of a media demand.
And that's all.
I'm sure you would
love this feature
in your favorite
podcast web app.
Now, here's another key
part of a perfect media user
experience.
And we code it rotate
to full screen.
As user rotates
device in [INAUDIBLE],,
let's be smart and
automatically request
full screen on the video to
create an immersive experience.
Note that video with
no custom control
get this for free, like
Global Play, for instance.
So how does that work?
Let's use the screen
edition web API,
which is certainly not yet
supported everywhere, and still
prefixed in some
browser at that time.
Thus, this will be another
progressive enhancement.
As soon as you do take the
screen edition changes,
the video is full screen
if the browser window
is Netscape mode.
If not, exit full screen.
That's all.
Eight lines of JavaScript
and all of a sudden,
you can make the media user
expand significantly better
on the mobile web.
What about out of
focus media playback?
When you detect your web page
or video on your web page
is not visible
anymore, you may want
to update your analytics
to reflect this, or pick
a different video track
of a lower [INAUDIBLE],,
post the video, or
even show some always
on top custom controls the user.
And to illustrate this,
let me share with you
two APIs you can use today.
With the pages
[? EBT ?] web API,
you can detect the current
visibility of the page
and be notified of BCBG changes.
This code simply post a video
when the page is hidden.
This happens when the lock
screen is active, for instance,
or when user switches that.
As mobile browser now of it
controls outside of a browser
to resume a post
video, I recommend
you set this behavior only if
user is allowed to play media
in the background.
And furthermore, when
the page is hidden,
video with no custom
controls are automatically
posted on Chrome for Android.
With the new
intersection of the API,
you can be even more
granular at no cost.
This web API lets you know
when an observed element enters
or exits the browser view part.
In this code, the visibility
of a small mute button
is toggled based on the
video visibility in the page.
If the video is playing
but not currently visible,
the button will be shown
in the corner of a page
to give user control
over video sound.
Here's a tip for you.
If you have lots
of video in a page,
and it is using an
intersection observer
to pause or mute
offscreen video,
you should consider
resetting the video source
to null instead, as this will
raise significant resources
in an instance call case.
We've covered a lot.
So a great example of all
these features is voot.com,
was launched earlier this
year a really cool media PWA.
So, please, welcome on stage
Rajneel, who will share with us
how it's going and
the future plans.
Rajneel, they're yours.
RAJNEEL KUMAR:
Thank you Francois.
Hi, I'm Rajneel.
I am the head of
Product and Technology
at Viacom18 in India.
We have had a great journey.
We launched in May last
year our OTD product,
which is called Voot.
Today, we are at about 26
million monthly active users.
And everything they
spoke about, about how
to use the various APIs they've
been releasing this year,
we have been able to use it.
And I think we have seen
some significant improvement
to a progressive app
experience by itself.
When we launched Voot for
the progressive web app,
we actually called
it Voot light.
And that is a product
that we are now
focusing on extensively.
The business metrics of what
we have seen as an uptake
has been phenomenal.
But I want to share a
little bit about why
we thought it was important
to make a progressive web app.
India's a large country.
And as we reached, you
know, the 26 million
scale, which we have reached
right now, what we realized
was that there were
a lot of devices
which were under-powered.
They did not have
enough memory on them.
And they still wanted
to access the videos
that they wanted to see.
And we realized that
one of the best ways
to do that was to be able to
do something which did not
require them to take space
from the phones from an app.
And that is where the
progressive web app came in.
One metric that I
wanted to point out here
was how our entire distribution
across the web ecosystem
has changed after we got
the progressive web app on.
It has gone for about 20%
of daily actives to 50%.
So that's a number we
see that has helped
us get through traffic in.
And obviously,
the marketing guys
are really happy, because
the cost of acquisition
has gone down.
I will be speaking today about
two different offline examples.
But the first I
want to speak about
is the one that we are doing
on progressive web app.
India has about 1
billion phones, but only
about 260 million 4G
and 3G connections.
Offline is a way
of life for users
to be able to access content
and be able to play it later.
Now, the internet access
might be available
to these users for the
biggest parts of the day,
maybe at office, maybe
when they are at a friend's
place or a coffee shop.
So we have taken the experience
of offloading from the apps
and really brought
it to the browser.
So user can set
something to download,
and when the
download is complete,
he gets a notification
and he can
go off and be able to
watch it at his leisure.
To explain a little about
how we have achieved this,
I would now like to call
Arik from our technology
partner, Kaltura.
Arik.
ARIK GAISLER: Hi.
My name is Arik Gaisler, and I'm
the VP technology at Kaltura.
Kaltura's an end-to-end service
provider for OTT experiences,
powering large media companies
such as Voot's OTT platform.
At Kaltura Engineering,
basically we
have three pillars on which
we measure our success.
First off is
performance, providing
a top notch performance
across any device,
across any platform for all
of our customers, users.
Second is accessibility.
Providing our
customers and users
the power to access
content wherever
they are, across any
device, whether they're
online or offline.
And third, the user experience.
Providing a consistent,
feature-rich user experience
for our customers, across any
device, across any platform.
All of this while
keeping in mind
that most of our
customers, such as Voot,
require us to support
premium great content, which
usually requires some form
of content protection.
As such, PWA is a
natural component for us
to use to provide
all three pillars.
For instance, up until now,
providing offline content
support for content protected
content for premium grade
content usually required
using a native application
on top of XL player.
By utilizing PWA
capabilities, we
can offer the same
experiences using a simple web
application--
mobile web application.
So now let's look at
a typical download
experience on mobile web app.
So first off, of
course, the user
will request to download a
specific piece of content
from the web app.
The web app, in turn, will
go to the CDN or origin,
download the
respective manifest.
If the content needs to
be content protected,
there will be another request
to the license server,
providing information on the
user, the content, the device,
for the license server to make
a decision regarding what kind
of license to provide for that
specific piece of content.
And now comes the
interesting part
where the application will
pass on to the service worker,
basically, a list of files to
download in the background.
And the service
worker in turn will
start downloading these
files from the CDN
or origin while not blocking
the application at all.
Once this fetch is
completed, a notification
will go out to the application,
which in turn will,
of course, notify the user.
But now the content is basically
available for offline playback.
So now lets look a little
bit at some debugging tapes
and some experiences
we had along the way
developing the PWA app.
So first of all, keep
in mind that if you
check the bypass for
network check box,
this basically means that the
fetch event on the service
worker will be skipped.
It will not be invoked.
Another thing to remember
is update on reload
is very good for
developing mode.
It's very helpful.
Just make sure to uncheck
this when testing, of course.
Specifically regarding
background fetch,
today, you still
need to use a flag.
This flag is experimental
web platform features flag.
Another thing to remember,
and to keep in mind
while writing your code, and
we'll see a snippet explaining
this in the next
slide, is basically
if a single fetch
listener fails,
the rest will basically
now be invoked.
You need to keep this in
mind while developing.
Developing a progress bar
is very easy, very simple.
Basically, a service worker
can update the application
on its progress within the app.
Basically telling
the application
how much of the content
itself was actually
downloaded already.
So this, together with
the total file size,
basically, you can create your
own progress bar, basically,
telling your users how
much of the content
was already downloaded.
Now let's look at some code.
The first snippet is
basically the invocation
of the background
fetch fetch API.
You can see a very
simple wrapper function.
The first thing we do is we
will insert into the Index DB--
the IDB--
basically, entry, representing
that downloaded asset,
that asset that's
going to be downloaded.
This will be used later on in
case a service worker gets torn
down or failed for any reason.
Basically, we still have a
record of that downloaded asset
to re-invoke or to provide
some information to the user.
The next call is basically
to very simple invocation
of the background fetch
fetch API, providing an ID,
and providing a list
of assets, of course.
This next snippet is basically--
this is the event
that gets fired out
once the background
fetch is finalized.
This is the event.
So the first thing
we'll do, of course,
we'll retrieve the
Index DB metric
that we just put in the--
just uploading inside the IDB.
Then we basically correlate
that back to the background
fetched assets in order
to make sure that we're
talking about the same assets.
And the next step
is basically putting
the downloaded fetched
asset into the cache.
And this basically means
that now the conduit
is available for offline
playback for the user.
Basically, what will happen
now after these functions,
basically the service worker
will notify the application
that the background fetch
has finalized, has finished.
The Index DB will be
cleared of the entry
that we no longer need,
and the service worker
will be cleaned up.
You can see a very
simple workflow
for a very cool experience.
Now I'd like to call
back Rajneel basically
to talk to us a little bit
about another very cool PWA
experience for Voot.
SPEAKER 3: Now I want to tell a
different story about offline.
India's a large country,
and we know that there
are about 1.3 billion people.
One of the main challenges is
that because of the geography,
a lot of places still don't have
stable 4G networks that people
can really stream content from.
The other challenge, of
course, is that the data costs
have been really high.
Over the last year, the
costs have come down
with the launch of
a new 4G network.
But still, data costs money.
We looked at solving the
problem in a different way.
We said, what if we did not
use internet to really get
the content to the people?
Marasha is a state which has
about 112 million people.
And we tied up with the state
transport corporation there.
And they have
about 17,000 buses.
What we looked at doing was
launch a part called VootGo.
So what you see
here is essentially
a bus station where
you can sticker out
and say, log on to VootGo.
When you enter the
bus, inside there
are stickers telling the
users to be able to access
by typing VootGo.
What would happen is that they
would find a Wi-Fi network,
and we created these
propriety boxes which
were placed inside the buses.
These boxes can store
about 200 hours of content
and can be refreshed, you know,
every time the bus goes back
into the depot.
And the users launch VootGo.
We customize the UI
to make it extremely
multi-lingual and
easy to understand
for people who might find
English as a challenge, which
is a lot of times.
So we launched
with two languages.
One is in Hindi, and
another called Marathi.
And we created a dedicated space
for kids content also in this.
Today, we're running this
on about 10,000 buses.
We have seen some
amazing results.
We have about 40 minutes
of content consumption
per user happening.
And we see about
nine users per trip
connecting onto the
platform and using it.
One cool thing about
this is that when
we reach scale of
about 17,000 buses,
we'll be servicing about 6.3
million people every day.
6.3 like people every
day, which essentially
means that we will be doing
about 2.3 billion passengers
will be coming onto the bus.
Now that's a large scale
deployment of a product
to get content out.
And this, of course, remains
completely free for the users,
because this is
advertising driven.
So what we're excited
about is creating
the best of internet in a
non-connected environment,
being able to deliver
content at scale,
and creating a completely new
business model for the media.
So that's what I've had
to share about something
that we're doing on VootGo.
With that, I'd like to
called John back on stage.
Thank you.
JOHN PALLETT: Thanks, Rajneel.
Well, as you can see, there are
a lot of great opportunities
out there for mobile web media.
And we've listed some of the key
APIs on the screen right now.
For those of you taking pictures
in the audience, those of you
at home, hopefully this is
a good reference for you.
A couple of notes on the APIs.
Background fetch is
still in development.
It is available in
Canary right now,
aiming to release
early next year.
And as mentioned,
it is something
that you need to turn on the
experimental web features flag
if you want to try using it.
Offline persistent license
support is in Chrome 62
for Android.
And its coming in Canary 64
soon for desktop as well.
So if you're doing
protected content offline,
you'll need that feature.
A couple of things we
didn't mention in this talk
but also worth
looking at-- if you
want to send media to
a television, please
do look at the cast API.
It's easy to add to your site.
And for anybody who has not
yet seen HDR video, VP9 10-bit,
tomorrow, at the
demo pod, we will
have that playing
on an HDR screen,
so that you can actually see the
future, some of the great video
quality, both in
terms of dynamic range
as well as wide color gamut.
So this is all coming.
It's really exciting stuff.
Ultimately, the
web is more ready
and more capable for media
playback than even a year ago.
And so it's been
great for users.
It's great for sites.
Because it makes media just
a single click or a URL away.
We're excited to see
continued growth in this area.
So with that, thank
you for your time.
Myself, Francois, we'll all
be available to take questions
afterwards.
And we look forward
to seeing you there.
Thank you.
