[MUSIC PLAYING]
SAM RICHARD: Thanks for sticking
with us to the end of the day.
The web is an incredibly
powerful platform,
and with its reach and the
installability of PWAs,
it's ideal for cross-device
software delivery.
But it can't do everything
that native apps can do.
We want to change that.
We want the web to do
everything that native apps can,
and be more secure and
user-friendly while doing it.
My name is Sam Richard, and I'm
a developer advocate for Chrome
OS, and I'm here to tell you a
story of software development
on the web, and the work
Google, Microsoft, Intel,
and others are doing to build
the future of the platform.
This is the story
of how we're looking
to bridge the native app gap.
The story for me starts
fall of last year,
setting up my new computer.
I had previously
used a combination
of native apps and command-line
tools to optimize images.
Then I tried Squoosh.app,
a PWA from the Chrome Dev
rel [? team. ?] It
worked on my Chromebook,
it worked on my Mac, and
it worked on Windows.
It's just a website.
But with a simple
install prompt it
found its way to
my app launchers,
and replacing all
of my native tools.
To me, it became
clear that PWAs should
be the future of
software delivery.
They're built using
modern APIs, enhanced
to deliver native-like
installability, reliability,
and capabilities, all while
reaching anyone, anywhere,
on any device.
There are still
some gaps, though.
The web is super powerful
in its own right today.
You can build a hyper
local video chat
app with WebRTC, geolocation,
and push messaging.
You can make that
app installable.
You can add video
effects with Wasm,
and you can even bring
it into new realities
with WebGL and WebVR.
But there's still gaps in
what we can solve with the web
today.
In the worst case, this
means that developers are not
building for the web at all,
or they're relegating it
to a second-class experience.
For those that do want
to build for the web
but need the
capabilities of native,
they're forced to bundle
web apps in native wrappers.
This often results in
developers effectively
shipping their own
custom browsers to users,
exploding the size
of their web apps
and forcing them to take on
the security and maintenance
burden of both
keeping the browser
and their native
wrapper up to date.
These native wrappers also
lose the reach of the web,
forcing them to choose
what devices and operating
systems to support,
often requiring
different native
wrappers, all of which
further expands the security
and maintenance burden.
Filling these gaps is the key
to fixing this and enabling
the web to be the software
platform of the future,
like we believe it to be.
What does this future look like?
What do you think is missing
from the web platform today?
What is the web you want?
I'm going to tell you
two stories of two
fictional developers,
Quinn, working
on Fugujournal, and Sawyer,
working on Fuguedit.
Both Quinn and Sawyer want to
solve problems for their users
that have usually been
outside the scope of what's
possible on the web.
As you'll see, both will
leverage new capabilities
that change what's possible
to build their applications.
We start with Quinn, the lead
developer for a social media
startup, Fugujournal.
Social apps on mobile
devices have been hard,
because they need to be
tightly integrated in order
to be loved by their users
Quinn thinks to themself.
Our users want to start a
journal entry from anywhere
on their device,
and they want to be
able to share those entries
to other social accounts.
They also want to collaborate
on entries with their friends,
so they need to be able to
invite them to do so easily.
Until recently,
these capabilities
would seem to rule out
building a web app.
They just simply didn't exist
Both Web Share and the new Web
Share Target V2
and Contacts APIs--
that calculus has
changed, and Quinn
can build a progressive web
app, instead of a native app.
Let's see what Quinn
is looking to do.
After a user navigates
to Fugujournal,
they should be able to install
it to their home screen.
Once installed, they should
be able to find media, images,
video, and audio from
throughout their phone
and share it to Fugujournal.
Users should then be able
to share any entry using
the native share sheet of their
phone to another application.
Finally, they should be able to
find contacts from their phone
to invite them to collaborate,
sending off invitations
once they verify their
contacts' information.
Diving into how Quinn
can make this work,
let's start by looking
at Web Share Target V2.
This allows installed PWAs to
both register as a share target
and describe what
can be shared to it.
To add it to
Fugujournal, Quinn needs
to update the web app manifest
to include a share target.
You can only have one share
target per web app manifest.
And there are three parameters--
action, which is the URL
to open when shared to.
This should be pre-cached
when your service
worker is installed, so
that it's always available.
Method, which determines how
to share items to your PWA.
If you want to use
POST instead of GET,
you need an ENC-type parameter
as well for encoding.
And then params, which is
what is allowed to be shared
and what those items
should be called.
Each of the parameters
inside of params is optional.
You can have title, you can
have text, you can have url,
and you can even have
an array of files,
each with their own MIME
type and a different name
for those MIME types.
When shared, the data
is sent to the action
URL as form data that can then
be parsed and used as needed.
A neat trick you can
do for a share target
is to register a route
directly in your service worker
to handle the request.
This will let you use it
even without a back end.
Here, we're importing work
boxes, route register,
and registering our
service or our share target
as a route, a post
route, calling
shareTargetHandler function,
which looks like this.
The shareTargetHandler
is an async function
that takes in the event
and awaits the form data,
and peels off the
media files from that.
You can then do whatever
you'd like with these files.
You can cache them.
You can send them somewhere
with a fetch request.
You can even make use of
the rest of the options,
maybe serving a page with
some query parameters
for the other shared items or
storing the data and pointers
to the media in indexed DB.
Once the journal entry
is in Fugujournal,
Quinn's users need to be
able to share it back out.
To do so, when the
Share button is clicked,
Quinn checks to see if
navigator.share is available.
If so, it'll use the
native share sheet.
If not, we can fall back
to custom sharing options.
To activate the
native share sheet,
Quinn calls
navigator.share, a promise,
and passes it at least one
of the optional parameters--
title, which is the title
of the item being shared;
text, which is the
text to be shared;
the URL that is being shared--
and note, this doesn't
need to be the URL
that you're currently on.
It can be any URL.
And then an array of files
to be shared out as well.
With both Web Share and
Web Share Target in place,
Quinn is ready to test
sharing to Fugujournal.
They start by finding a file
to create an entry from.
Opening the file, they
tap on the kabob menu,
hit Send, find Fugujournal
in the share sheet,
and then tap it.
This opens the image
in Fugujournal,
posted and ready to go.
They then click the Share button
from an entry, and, hey, look!
Android's native
share sheet pops up,
ready to share Twitter's
PWA has also implemented
Web Share and Web
Share Target V2,
so Quinn can share their
entry over to Twitter.
With a tap, Quinn's brought
over to the Twitter PWA,
and their post is
ready to be tweeted out
Quinn now turns their attention
to the contact picker.
Contact picker is
also a promise,
so we're going to wrap
it in an async function.
When the function is run,
Quinn checks for support
by making sure both
navigator.contacts
and window.ContactsManager
exists.
If not, the user will get
redirected to a manual contact
entry form.
If it does, then the contact
picker will be launched.
The picker takes two
parameters, props,
which is an array containing
the properties to be requested,
and options, which is an object
specifying if multiple contacts
can be selected.
The results are an array.
It's either empty if no
contacts were selected
or the contact picker
couldn't be launched.
Otherwise, it's
full of contacts.
Quinn then takes those results
and passes it to a function
to populate the invite form.
Quinn gives the new
contact picker a test run.
Navigating to the
Invite screen, Quinn
clicks the Invite Friends button
and is presented with a contact
picker.
Now, there's been
lots of thought
around user security put
into this contact picker.
Apps aren't granted
permanent access.
It's on demand.
Users also aren't
required to provide all
of the information requested.
They choose what's
returned from the request.
Quinn decides to
select a few contacts,
but only share their
name and email address.
Once selected, the contacts
now show up in Fugujournal.
And with that, Quinn
can invite their friends
to come try their
newly launched site.
And so can you.
Go try it yourself.
On your Android device,
go to fugujournal.web.app,
add it to your home screen, and
try out everything you've seen,
inviting friends, adding
content, and sharing it out.
While the Contacts API
isn't fully launched yet,
I've enabled an origin
trial on Fugujournal
so you can see how it works
without a special version
of Chrome or toggling
any Chrome flags.
As a demo site, it's not
hooked up to any back end,
so no information
will be collected,
and all files shared
are stored locally.
We now move from
Quinn to Sawyer.
Sawyer is developing a
new desktop-class web
editor, Fuguedit, and
heard you like websites,
so he wants to
build you a website
to let you build websites.
Sawyer has similar
concerns that Quinn had.
Editors are hard to build.
They really need
to be seamlessly
integrated into an operating
system to feel useful.
Sawyer thinks editors
are complicated.
Our users want to be able
to pick a file or folder
to work on and just work on it.
Users get attached to
their favorite editors,
so they'll want to be
able to quickly open
files from their home
screen or their file
system in the editor.
These requirements,
much like Quinn's, used
to be impossible with a web app.
Previously on the web, in
order to edit files or folders,
first you needed to upload
them, then make your edits,
then download those files again.
You couldn't just
change them in place.
It was pretty cumbersome.
And opening a file from the
file system into an app--
simple for native apps,
previously impossible for web
apps.
Just like Quinn, Sawyer is
going to turn to new APIs.
This time it's the Native File
System API and the File Handler
API.
Let's see what
Sawyer's looking to do.
Navigating to Fuguedit
on their laptop,
a user sees a notification
in the omnibar
that they can install it.
Clicking it, they
install Fuguedit,
and it's now available alongside
all of their other applications
on their computer.
Once installed, they find
a file on their computer
and open it directly
into Fuguedit.
They can then make changes
that file using Fuguedit,
save those changes,
and then confirm
that those changes are real
by looking in the terminal.
Finally, they can open
a whole folder's worth
of files in Fuguedit, opening
individual files from the file
tree as they need
to work on them.
Sawyer is going to start with
opening a file from the file
system.
The native file
picker is a promise,
so we're going to use async
when the Open button is clicked,
or an async function.
The first thing
Sawyer does is check
to see if
chooseFileSystemEntries
is available.
If not, it'll fall back to
other file picker options.
If available, Sawyer's going to
call chooseFileSystemEntries.
One option is included
here, accepts,
which here will narrow
down what kinds of files
can be selected by MIME type.
We can also choose to
allow multiple files,
open directories
instead of files,
or create new files
instead of opening
a file that already exists.
Selecting the file is step one.
It provides basic
information, the file's name,
whether it's a directory,
and whether it's a file.
Calling getFile lets
Sawyer know when
it was last modified, its
size, and its MIME type.
Finally, in this case calling
.text is the final step,
and it will read the stream
of the file as a UTF-8 encoded
text, kind of like how
fetch's text method does.
If users open a file, they need
to be able to save it as well.
Sawyer is going to make
this feel more native,
so they're going to override
Control or Command-S to save
the file, instead of
saving the web page.
This will kick off
writing a file to disk.
To write a file, Sawyer
first creates a writer
from the file handler.
This creates a temporary file.
And they write the
contents to the temp file,
starting at the zeroth
character and including
all of the content.
Finally, they close
the temp file,
which updates the
original file on disk.
Loading up Fuguedit
on a Chromebook,
Sawyer has got two options,
open a file or open a folder.
Open a File feels very much
like the file upload form
that they're already used to,
and once opened, it pops in.
Opening a folder, however,
is a different story.
Sawyer selects a folder
from their Linux environment
on their Chromebook and sees
a notification warning them
that they're about to
grant access to a site
to view all files in that
folder until they close the tab.
This is a double opt-in, because
much like the Contact Picker,
this API was designed
with security in mind.
With a folder open,
Fuguedit's adapted
to include a folder
tree of all the files
and folders in that directory.
Sawyer can now move
around the folder tree,
opening files as they go.
Sawyer opens the README
file and makes some edits.
This was edited from a PWA.
Sawyer then hits Control-S
to save the file.
They get a notification telling
them that if they agree,
Fuguedit's going to have
write access to this file
until the tab's closed.
Again, an async block
to ensure the user
doesn't accidentally
give permission
to this functionality.
After saving once,
Sawyer checks to see
that the changes have stuck
by [? catting ?] the file
in their terminal.
The file's changed.
Great.
Going back to Fuguedit, Sawyer
makes some more changes,
presses Control-S
again, and then
checks again to see that
the changes have stuck.
Now that they're ready to
use Fuguedit to edit Fuguedit
itself, just one
more thing to do,
opening files from
the file system.
Doing so is similar
to what Quinn
did with Web Share Target.
Sawyer needs to update
their web app manifest.
The first thing
they need to do is,
they need to add a file handler
option to the web app manifest.
This allows Sawyer to specify
the URL, which should be used
when opening a given file type.
Sawyer can specify multiple
file handlers here.
One URL can accept
multiple file types,
and for OSes that work with
MIME types, those will be used.
Otherwise, it'll fall back
to specific extensions.
This also means you can
use custom extensions
with your PWA.
The final thing
that Sawyer needs
to do to get the file opened
is to actually get it.
To do that, on load,
they set a consumer
for the new launch queue.
The launch parameters
in the consumer
include an array of files
that are being launched with.
They pick off the first file and
await opening it in Fuguedit.
Loading up Fuguedit
on their Chromebook,
we go to the Files app, find a
file in our Linux environment
that Fuguedit can work with,
and open it with Fuguedit.
Ta-da!
Opened and ready to work with.
And again, you can
try this out yourself.
On your computer, go
to fuguedit.web.app,
install it, and try to
open the files and folders
and save them as
you normally would.
The Native File System API
is also not fully launched
yet, but just like
the Contacts API,
I've enabled an
origin trial, so you
can see how it works without
a special version of Chrome
or toggling any flags.
File handling isn't
available as an origin trial
yet, so you need to
open it up in Chrome
on either a Chromebook, a
Linux, or a Windows computer.
Go to chrome://flags and
enable the file handling API.
Support is coming for
Macs, and implementation
on other operating systems
is still kind of ongoing.
So you'll likely
run into some bugs
while we finish our
initial implementation.
If you want to see the source
code for both of these apps,
check out the Bridging
the Native App Gap repo
on github.com/chromeos,
one word.
While Quinn and Sawyer
may be a glimpse of what's
possible in the
future, teams are
starting to use these
new capabilities today.
But to do so, they
need to be confident
that they can use
these APIs securely.
These APIs that we're
exposing are really powerful,
which can be really scary.
And a lot of thinking
has gone into ensuring
that going to a
website continues
to be a safe proposition.
They're all being designed
with a user-first permission
model, where all permissions
are clearly defined and easily
revocable.
In fact, security and
privacy is so important
that if a capability can't be
made private, safe, and secure,
it won't be exposed.
They're also being developed as
part of an open standardization
process.
Tomorrow morning,
there's a whole talk
on this, intended to explain
demystifying the blank shipping
process that you should watch
to learn more about that.
All the APIs I've
talked about are
in the stage where
they're publicized,
and we're actively looking for
feedback to iterate on them.
Or, we're looking for
public support and for you
to try it out today using the
origin trial or the latest
stable Chrome release.
So with these APIs in the wild,
who's actually using them?
Goibibo is a leading online
travel booking brand in India.
While most users
book for themselves,
15% of their bookings
are for other people.
They've recently implemented
the Contact Picker
API to allow users
to choose someone
they'd like to book travel for,
smoothing out that process.
They're also looking into
other ways of incorporating
the Contact Picker API,
including sending referral
links and codes over SMS
and incorporating goCash+,
their system for rewarding users
when contacts they have synced
make a booking.
Google Earth Studio
is a browser based
animation tool for Google's
3D and satellite imagery.
It allows users to program
their camera's flight path
and then render that animation
as an image sequence.
It used to rely on the
old Chrome File System API
to store files during render.
This required generating a
zip of sometimes gigabytes
of images that was then
downloaded to the Downloads
folder.
This posed significant
limitations
on the length and size of
animations they could make.
It also caused confusion and
frustration for their users.
They couldn't find
the generated files.
They couldn't save to
an external hard drive,
and they would get out of
space errors when lots of space
was available.
The Native File System
API solved these problems.
When users start, they choose
the folder to save the file to,
and accepting the prompt
to allow the site to view
files in that selected folder.
Notice how the folder
just has the file
we started with, nothing else.
Once the user starts
to render, they're
prompted to allow Google Earth
Studio to write to that folder.
When allowed, Earth Studio
creates a working directory
it will put the rendered
images in, as well
as make a copy of the
camera's flight path.
When the images start
to render, they're
put in a new folder named
Footage that is easy for users
to find and keep track of.
By directly choosing the
output folder to render to,
there's no zipping, no confusion
as to where the files are,
and all of the user's
disk space is utilized.
In short, it's a dream come
true for the Earth Studio
team and their users.
Finally, I want to
talk about avrgirl.
Avrgirl is an
open-source project
that talks to the
Arduino bootloader
and writes a program from
the computer's hard drive
to its memory.
It's written by Suz
Hinton, aka noopkat,
an open-source
embedded developer
Suz does a lot of
code streaming,
and at the end of
September, showed
off something super
cool that she'd
be working on for avrgirl.
By leveraging another one
of these new capabilities,
we've been working on,
the Web Serial API, Suz
was able to program her
Arduino Uno to blink straight
from her browser.
Let's watch.
[VIDEO PLAYBACK]
- So this is the Upload-o-matic.
What it does is, it
takes pre-compiled code
for an Arduino
[INAUDIBLE],, and it's
able to use Web Serial
in order to upload that.
So I have some
pre-compiled code that I'm
going to try this out with.
I have an Arduino just
above me that you can watch.
So I'm going to pick
my Arduino Uno board.
I have some files here.
Let's go with blink,
so that we can actually
see that the code did
upload, and then we
should see a light
blinking when that's done.
I'm going to click Upload, which
allows me to give permission
to access my serial device,
which is my Arduino here.
And as soon as I
click Connect, that's
going to start uploading.
And then you have it.
The light is now blinking.
And as that was
being written, it
was dumping out all
of the pages of memory
that it was actually
writing to that device
as well, which is pretty cool.
[END PLAYBACK]
SAM RICHARD: I agree.
It is pretty cool.
[LAUGHTER]
So what's new, and what's next?
What Twitter, Google
Earth, Goibibo and avrgirl
have been able to do
with his new capabilities
already is pretty impressive.
But now it's your turn.
What new capabilities can you
add to your web apps today?
What can you experiment
with, what is in development,
and what does the future hold.
So things that have already
shipped as of Chrome 79--
we've got Web Share, Web Share
Target V2, the Shape Detection
API, which is for barcodes,
face, and text detection.
And we've got
getInstalledRelatedApps
for web apps that
have established
a connection to a native
app through something
like a digital asset link.
They'll be able to query if
that related app is already
installed.
There are also a bunch
of stuff in origin trial.
So things we've
talked about already--
Native File System, Contact
Picker, File Handlers,
and Web Serial.
We're also going to have
a Badging API for app icon
badges, as well as Wake Lock,
to prevent the screen from going
to sleep, an SMS Receiver, to
enable SMS one-time password
flows.
Periodic Background Sync,
to run a service worker
at designated intervals
while online, for instance,
to update the status or
content in the background; Web
HID, or Human Interface
Devices, great for getting
the most functionality possible
out of things like gaming
controllers; Notification
Triggers, which
are notifications based on
events like timers or location,
instead of push events; and then
Back Pressure for Web Sockets,
to improve handling for
large amounts of data
over web sockets.
What are things that
are in development?
Well, Font Enumeration, to get
a list of local system fonts;
Font Table Access, to allow
TrueType and OpenType table
access for fonts, to allow
custom font rendering; Raw
Clipboard Access, to
extend async clipboard
to allow unrestricted access.
And two that I'm really
looking forward to,
Screen Enumeration
and Window Placement.
Screen enumeration
makes applications
perform well across
multiple displays
with different properties,
like presentation
apps or financial dashboards,
and then Windows Placement
allows you to create,
manipulate, and enumerate
windows for an
application, including
moving windows as a
group and moving windows
to specific screens.
Finally, what's
coming in the future?
So we're looking into things
like Service Worker Launch
Events, which control where
links open, if they open in
a PWA, a tab, or a window; a
low-level audio API; User Idle
Detection; a Tab
Application mode for PWAs;
Native Sockets; Scheduled
Tasks; and being
able to control
connected cameras.
If you want to keep up to
date with all the work that
we're doing, go to
goo.gle/fugu-api-tracker.
And it's a really fun
URL to say out loud.
The status of each
of the APIs that
are being worked
on or considered
is there; where the
API is targeted for,
mobile, desktop, or both; and
links to the implementation
tickets for each capability.
We also need you to try these
APIs out and give us feedback.
If you want to explore
more, come find us over
in the sandbox
area, or we've got
a booth dedicated to
these new capabilities,
including code labs for
you to try out yourselves.
I will be also be over
at the Chrome OS booth.
You should also sign
up for origin trials
so you can try these out
with your real users.
Origin trials give
you the ability
to try out experimental APIs
with real users of your site,
without needing them to
change their Chrome version
or toggle any flags for
a limited period of time.
Features in origin
trial have what's
called one week of
guaranteed breakage.
This means that
you're guaranteed
to have a feature
not work for a week
before the feature
becomes stable,
as origin trials aren't meant
to be early-launch mechanisms.
Finally, origin
trials you can also
give us direct feedback
on their implementation
and your usage of
these features.
Setting up an origin trial
can be done in four steps.
Step one, choose
your origin trial.
Step two, register your
domain for that origin trial.
Step three, get the token
for your origin trial.
And step four, add a metatag for
your origin trial with that key
that you've gotten.
With that, you can
try out the feature
you'd like with your real users
and provide us with feedback.
Finally, what's missing?
All of these APIs, we're
developing because we're
being asked for them.
Participating in origin
trials and giving feedback
there is great way
to help us shape APIs
that we've started
working on, but we
want to know what you need
before we get to that stage.
Let us know what you need.
Go to webwewant.fyi
to let us know
what you'd like to see
added to the web platform
to help you bridge
the native app gap.
Thank you.
[APPLAUSE]
[MUSIC PLAYING]
