[MUSIC PLAYING]
CHENEY TSAI: Hey, everyone.
My name is Cheney.
RAMYA RAGHAVAN: And I'm Ramya.
CHENEY TSAI: So we work
on a technical consulting
team at Google, and we
get to see companies
all around the world push the
boundaries of the modern web.
RAMYA RAGHAVAN:
And while you might
have heard some
great sessions today
talking about different
modern web technologies,
we are here to focus on the
real-world challenges faced
by e-commerce companies
in implementing
these technologies.
CHENEY TSAI: So since we're
talking about the real world,
a real-world metaphor.
Maintaining a website is a
lot like maintaining a house.
This right here is the
house that I grew up in.
It's probably not like the
house that you grew up in,
just like the website you work
on is different from, say,
the person over there.
But some things are the same.
You have maybe a
place you sleep.
Maybe there's a dining room.
Then something
breaks, like the AC,
you might say,
well, I can fix it.
And your friend might
say, no, you should just
call somebody, all right?
Over time, when
you moved in, when
you moved to that
new platform, it
might look like the room
over here on the left.
But technical
debt, feature creep
because Black Friday and the
holiday season is coming up,
you're trying to get things
in, and you get kind of messy.
And I think that a house
is a great metaphor
because it's like an
e-commerce site these days.
It's not just about landing
in one place and you're done.
It's about the journey.
You're going from room to room.
You're going from a
page to another page,
from the home page to the search
page to the checkout page.
And if the room is like this,
you get kind of confused.
You don't want your user to feel
confused and get lost in it.
So it's time to remodel.
Maybe you need to
upgrade your foundations.
Obviously, to
remodel, we've talked
about a lot of different
things throughout today
and other talks,
things like that,
and like all tools
in the tool box,
things like Progressive
Web Apps, AMP,
Google sign in, Google
Page-- they're great tools.
But just like I
can't go to the store
and buy a power drill that
I saw a great commercial of,
and be like, I'm going
to nail this remodel.
I could very well
destroy everything.
So like all tools,
just because they fit--
they're kind of
like puzzle pieces.
The two pieces fit, but you
might not be actually making
progress towards a big picture.
So what's the big picture?
So at Google, we like to
say, focus on the user
and all else follows.
And so we really think
that that full picture
you're building
towards, whatever
you choose to do in
the e-commerce journey,
you need to improve
user friction.
Think about if you're trying to
add service workers, if you're
trying to try some of the
cool, new APIs, and like, WebXR
and things like that.
How is your solution actually
improving the user's journey?
Because there's no
silver bullet here.
To talk about this is Ramya.
You had an example?
RAMYA RAGHAVAN: Of course.
Saatva is a company that
sells luxury mattresses.
And they wanted to ensure
that their users have
a very smooth and fast
online shopping experience.
They decided to use AMP as
a technology to enable that.
In their first iteration
of their product details
pages where they created the
AMP portion of that page,
the user had actually to
click from the AMP page
to the product details
page and then click again
to get to the cart page.
They realized that this was
not the best user experience.
So in their second iteration,
they changed the design such
that the user can see the
full details of the product
within the AMP page and be
able to click to add to cart,
and even see the cart
from the same page.
This resulted in a reduction
of friction for the user,
going from three taps to
two all in the same page
instead of three pages earlier.
The results were astounding.
Between the two
iterations of AMP,
they saw a 50% improvement
in conversions.
CHENEY TSAI: Thanks.
So it's not just AMP.
If you're creating things
like a Progressive Web App,
you might run into this too.
So here's an example
from Expedia.
Expedia has been working on a
brand new front-end platform.
It's four times faster on
slow 3G, on lower end devices.
It's really great.
And their CEO even recently
said in their earnings
call that speed benefits
deliver conversion benefits.
Usually, we'll say,
we're going to stop here.
Great case study.
Instead, we're going to go into
an interesting learning they
had as part of this process.
So Expedia, like many of the
sites that you might work on,
is a large e-commerce site.
They're not just going to move
from one old platform that's
clearly working over to
a new platform overnight.
They need to do this
in a piecemeal way
because maybe some teams
are on a new platform.
They have to socialize
it to other teams.
And so in doing
that, they actually
learned an interesting lesson.
Users value consistency.
So some users during
the migration process,
they landed on the
old experience.
And then, after
clicking around, you
get put into the new
experience that's faster.
And they might move back
and forth a couple of times.
That population, to no surprise,
actually probably did poorer
than if you just stayed in
one experience or another.
It's very much like,
if I was up here,
and I was wearing one brown
shoe and one black shoe,
you probably couldn't
take me quite as seriously
as you're taking me right now.
Hopefully, it's on
the serious side.
So I think users value user
journeys that are consistent.
So make sure your solutions,
not only in that page
you're building, but
in the entire journey
and how you're measuring it
is actually reducing friction.
Of course, what
I'm saying, though,
means that if your journey
needs to be consistent
all the way from, say, the entry
point for the search results
page to checkout,
you're probably
going to run into a
lot of challenges.
So when we work with a lot
of e-commerce partners,
we run into three
main challenges.
Number one is something
that many of you
probably think about,
organizational alignment.
Then you get to
technical approach,
and then finally, measurement.
And so we'll talk
about that today.
RAMYA RAGHAVAN: Of course.
Let's talk about
the first challenge,
organizational alignment.
When I talk to people
in e-commerce companies
and ask them, do you think
investing in performance
projects to improve user
experience will help improve
your core business metrics,
the answer is an immediate yes.
However, when it comes time
to prioritizing performance
projects, there's
always several hurdles--
lack of time, lack of
resources, you name it.
One of the key ingredients
that's missing here
is organizational alignment.
Performance is not just
an engineering priority.
Why?
Consider this.
Your content creation team
wants to generate great content
for the users to consume.
However, they
might create images
that are too heavy
or not optimized
because they are not going
through the image optimization
layer.
The marketing team
spends several dollars
to bring traffic to the
site and care deeply
about the user experience.
However, they might
also have tags
that are either old or
redundant or obsolete.
The design team cares very
much about that end-to-end
experience, but they might have
some really cool new features
that do not use
shared competence.
In each of these
scenarios, one thing
is clear, which is
that performance
needs to be a priority
among every team that
affects the website and
cares about user experience.
Let's look at two
approaches, a top-down
and a bottom-up
approach, in order
to create organizational
alignment.
And we'll do this using
two examples of partners,
1-800-Flowers, which is a
global flower brand, and then
JustFly, which is a
Canadian travel agency.
1-800-Flowers believed that
the modern marketing department
is really marketing
plus technology.
This is a very big
shift in mindset.
And what they did was to break
out of organizational silos
and create a
cross-functional team that
comprised of product,
marketing, design
development, and analytics.
This was led by the
CMO of the company.
And they were given a
very clear shared goal,
which was to drive
growth for the company
by investing in the emerging
product experiences.
This allowed this
cross-functional team
to work really closely to
invest not just in one or two,
but almost the entire spectrum
of modern web technologies.
And as you can see, this
yielded some great results.
Seeing a 50% uplift in
conversions does not come easy.
Let's look at the
other example, JustFly.
Even if you don't have
a top-down mandate,
you can use data to
kind of prove your case
and get that consensus.
JustFly did just that.
They started off with
bite-sized proof of concepts
and experiments and
got the consensus
amongst their management
and cross-functional teams
to invest in a [INAUDIBLE].
They started off with
this high-level goal
of improving the booking
experience on their site.
And they used data
and experiments
for just about everything, from
testing different GSA libraries
to testing different
checkout flows and design.
And also, they had [? at ?]
[? the ?] [? home ?]
[? screen ?] icon being
[INAUDIBLE] by a few users
to see how those users behave
differently from others.
And with all of this
data, they not only
got the buy-in from
product and marketing,
but they also got the head count
they needed to build the PWA.
And they merged the code
bases of two of their brands
so that they could
launch both of them
as a PWA in the same
time frame that it
would have taken for one.
Now, that's efficiency.
And as you can see, with a
35% improvement in bookings,
this is a great result for them.
All in all, whether you
use a top-down approach
or a bottom-up approach, it's
important to keep in mind
that organizational alignment
is key to not just prioritizing
your performance
projects, but also to gain
the right visibility
so that everybody
that cares about user experience
also cares about your project.
CHENEY TSAI: Well, once you
get the organizational buy-in,
you still have to
build the project.
So we're not here today to
talk about all different ways
to build a great Progressive Web
App or a modern web experience.
We'll share a little bit
of these successful traits
we see of good projects
we see out there.
One big pattern is
that we see success
in having a long-term
vision and then
having achievable goals
towards that vision.
To give an example--
so I was recently in
Spain with my wife,
and we visited la Sagrada
Familia over in Barcelona.
I think it's a great
example of this.
This has been under construction
for five generations.
And yet, it still adheres
to the original architect,
Gaudi's, original vision.
Despite that, even
though it's still
under construction to this
day, it opens its doors
to 2.5 million tourists a year.
It's actually very much like
some of the e-commerce sites
that you have.
On Black Friday, not only are
people going to check out,
you probably are running
like 2.5 million A/B tests
through some platform.
So I think it's
really interesting
to think about that.
And a lot of teams have
that long-term vision,
and they often tell
me that, yeah, we
want to get that
time-to-interactive
to under five seconds.
But oftentimes,
you find yourself,
we ran Lighthouse and we're
really far away from it.
And what happens is, well,
we'll get there someday.
And then there's nothing
achievable to get there.
It's just something you see off
in the distance in the horizon.
Oftentimes, they come back
and say, well, we're just
going to have short-term goals.
And this is great.
They have sprints or
everything to get quick wins.
And not to say quick
wins aren't bad,
but oftentimes, you kind of get
into this internal cycle of,
well, there's a deadline here.
It's something we can achieve
before the holiday season.
And then that's all you do.
You go in the circles or you
regress after it a little bit.
It's really important that you
have both a long-term vision
and a short-term
goal and you make
sure that they are
actually related,
that the short-term goals is
making meaningful progress
towards a long-term vision.
Let's make this a little
bit more concrete.
We all like to say it up on the
stage with the other speakers
that one of the visions
is to improve speed.
Obviously, what does that mean?
There's a lot different
ways to measure.
So let's go back to what I
was saying earlier from a user
perspective.
Improving speed might be to
focus on user navigations.
So when we say you want to
optimize your cold page load,
or maybe your return page load
to see if you're using caching,
or if you're comparing
single-page app
to multi-page app,
a lot of things
we're talking about that
ease, that latency between me
clicking on the user
interface and the response
in your UI of
something showing up.
So let's talk about
speed in that regard.
So to go back to the
house really quickly,
I think doors are a
great kind of example
of what navigation is.
An e-commerce site--
it's like a house.
You move from room to room.
You might come in through the
back door, the front door,
or something like that.
And throughout the
process, to complete
the journey in the house,
like to complete the journey
on your website, you
move from room to room.
On Thanksgiving, you're holding
the hot turkey in the kitchen.
Do you want to mess around
with the door on the left
here where you have
to find one key
and then grab another key to get
through the door in the back?
It's probably not
the door you want.
Likewise, there are
probably experiences
in your e-commerce journey
where you don't want the user
during sorting,
filtering, or searching
to have to kind of pay the
cost of that locked door.
There might be a good place for
you to have a door on the right
here.
I'm not saying that it's always
client-side routing or anything
like that, but there are
different techniques of making
that as seamless as possible.
So let's jump into three
examples from Walmart, eBay,
and Airbnb, who are in different
parts of this modern web
journey, and kind of dive
into how they're breaking down
this process.
Walmart's a great example--
extremely leading
e-commerce site,
and it recently made a North
Star Goal to reduce time
to interactive by 70%
on low-end devices,
such as the iPhone
6s and Moto G4.
And this is an extremely
aggressive goal to going back
to what I was saying earlier.
But I liked that they were
able to kind of make that goal.
So they organized a bunch of
product and engineering teams
to make meaningful
progress towards that.
And then in the last couple of
months, they got 25% of the way
there with improvement
on some pages.
But just because you make
good progress from 0% to 25%
doesn't mean just making
quick wins or the same process
is going to take
you from 25% to 70%.
But that work is important
because in that work they did,
they were able to show
significant business
improvement and use
that to socialize it
across their teams.
But what do I mean by
going from 25% to 75%
might not be quite as simple?
Take this.
It's not really my kind of
room, but let's say it is.
I could organize this room by
just taking some books out,
and reshuffling, and making sure
they're all in the same order.
But if I was going
to take that strategy
and do that for
everything in this room,
I don't know if I'll
make meaningful progress
and finish in time.
If I'm going to
clean up this room,
I might actually
say, well, look,
this is kind of how I would
like one shelf to look like,
but let's kind of take
all the books away,
move the shelves
around to actually get
a nice looking place.
Likewise, what Walmart's
doing is that because they
had a North Star Goal and
they made short-term progress,
they're now able to start
thinking about, well,
what other larger shifts
can we take to get
to the goal we want to do?
Is it building a
Progressive Web App?
Is it adding service workers
in some meaningful way?
I think we're really
interested to see
kind of what they're going
to work on in the next year.
Speaking of service
workers, eBay
is actually using one of the
service worker techniques
to improve user
navigation on their site.
So in one of the
earlier talks, we
talked about how you
can use service workers
to improve user navigations.
There's lots of different ways.
eBay is just using
one potential way,
and they're testing
this right now.
So they're doing predictive
prefetching of listing pages.
So for example, right now,
if I go to eBay, and say,
I'm looking for a camera
for Christmas for my dad--
and dad, you didn't hear that.
I don't think you're tuning in.
So right now, if I see a
camera in the search results,
I have to wait until I
click on the listing,
and then the browser makes
another HTML request,
and I'm waiting
for that as a user.
And depending on what
connection I'm in,
that could take a long time.
Well, eBay's like, if we could
predict with high accuracy what
you're likely to click
on-- you're going to click
on this camera right here.
What if we post message
to this service worker
that particular file you need.
And then the service worker
is pulling in the background
while I'm considering
which camera to click on
so that when I do
click on it, well,
it's already there on my phone
so that when I click through,
it's as instantaneous of
a navigation as possible.
Of course, what if you
don't have a service worker
architecture yet
already in place?
You're waiting on Q3
for that to be done.
Or maybe some of the quick wins
for initial page load might not
be there and you're
kind of stuck there.
Because you have a long-term
vision of improving user
navigation, you might run
into a wall at some point
of a short-term goal.
Because you have
that setup, you can
start thinking about
different ways of getting
to that long-term vision.
And this is kind of where Airbnb
is kind of thinking about.
So Airbnb is like, well,
sometimes rebooting
that JavaScript location
on every single page
might be a cost we don't want
every single page load to be.
So they saw that,
what if we can only
update certain
parts of the page,
say, in the search results so
that only that refreshes, only
that shows up when the
user clicks on something?
In doing this, they
actually measured
the user-perceived
speed of that load.
And they saw that loading
the search results
page in that manner was a seven
to eight time improvement.
And so let's say they
start diverting more user
traffic to take advantage
of client-side routing
so that now 88% of
search results loads
are leveraging the seven to
eight times speed improvement.
Because at the end
of the day, it's
the user's journey that matters.
Hopefully this is
a quick example
of how you can keep an eye on
long-term vision, wherever that
may be, set short-term goals,
and realize that there are
a lot of different
ways when you hit
different walls in those
short-term goals of getting
to that long-term vision.
RAMYA RAGHAVAN: Let's talk
about the third challenge,
measurement.
Let me ask you something.
Please raise your
hands if you've ever
worked on a project
with great potential
to improve user
experience, but the plug
got pulled at some point because
you couldn't prove the impact.
Anyone?
That's quite a few of us.
And this is certainly
not uncommon, right?
Even if you have a great
organizational alignment
and a good technical
approach to your project,
if you don't have the
right measurement strategy,
it could still fail.
Because of this, even though
this is the third challenge
we are covering here today,
this is one of the first things
that I've seen
successful companies do
in terms of thinking about
their performance project.
A good measurement strategy is
one that is shared, automated,
and actionable.
What do I mean by that?
It needs to have shared metrics.
To Cheney's earlier point, when
you have the North Star Goal,
you need to have
specific metrics that
are mapped to that North
Star Goal that are also
shared amongst people that care
about that user experience.
Having these shared
metrics is really crucial,
and you have to
define it in a way
that people understand
exactly what those metrics are
and how they are calculated.
The second piece here
is automated tools.
These tools are necessary, not
just to measure these metrics,
but also monitor them over
time so that you can gain
real good insights from them.
And once you have
those tools, it's
important to understand, what
are the actionable next steps?
Now that I have this
insight, what changes
can I make in order to
meet that North Star Goal?
Let's dive into shared metrics
a little bit deeper because it's
often not as well understood.
When I talk about mapping
goals to specific metrics,
we have these four
big milestones
when we talk about a
user journey in terms
of acquisition, engagement,
conversion, and re-engagement.
Let's focus more on
the conversion piece.
Let's say you really wanted a
seamless checkout experience
for your users.
One way to measure that is to
think about the average time
that it takes for, let's
say, 90% of your users
to go from that cart page to
the order completion page.
Understanding from
a user perspective
how easy or difficult
it is for them
to convert when
that intent is clear
is a good way, potentially, for
measuring whether you're having
a seamless checkout flow.
Another important point to
consider is third parties.
Third parties are an integral
part of that user experience
on your site.
And therefore,
they should not be
ignored when it comes to
measurement strategy either.
So one way you can hold
your third party accountable
is to say that the
server response
time should be under a certain
budget that you set for them.
All in all, talking
about budgets,
you've been hearing this
term of performance budgets
throughout various
sessions today.
A performance budget
is a framework
that allows you to understand
what changes represent progress
and what changes
represent regressions.
And it does take into account
the set of shared metrics,
and you have budgets for
each of those metrics, which
are shown in this automated
tool and made actionable.
Let's actually
take a look at how
three different
e-commerce companies
took this idea of a
performance budget
and implemented it
so well that they
have seen some amazing results.
Are you ready?
OK, Cdiscount is a French
leader in e-commerce
that had a problem that is
not very unlike many of us
in e-commerce.
They saw a big chunk of
their traffic on mobile,
but they didn't see that
the mobile sales were
at a similar ratio.
They realized that this is
not an easy problem to solve,
right?
So they broke
organizational silos,
and they empowered
their marketing team
to run deep analysis
and come up with a set
of shared best practices.
They then implemented these
best practices as budgets
within their tool.
And using the integration
between SpeedCurve and Slack,
they ensured that when
the budgets were not met,
automatic alerts would be sent
into the Slack channel which
was shared amongst all the
cross-functional teams.
By collaborating in making all
of these different changes,
they saw some really,
really good results.
Their mobile website
conversion rate
was 50% higher than their
native app, which has a 4,
4 and 1/2 star in
different app stores.
This is something that all of
us really can relate to, right?
It's not easy to get there.
Let's take another example.
Carousell, which is a
Singapore-based online
marketplace, used this idea of a
performance budget because they
really wanted their users in
southeast Asian countries where
the predominant
network speed is 3G--
they wanted all of those users
to have a great experience
on their site.
And so they started
off with this kind
of performance budget.
Note that the first line here
is about the critical rendering
path resources bundle
to be under 120 KB.
This is irrespective of
whether that bundle includes
first party or third
party because they
had this commitment
that that bundle needs
to be really small.
They were so intent upon
following this performance
budget that they
included this as part
of their continuous
integration process, such
that the code
could not be merged
until this bundle or
this budget was met.
In order to do that, it was
not an easy thing at all,
so they really had to
collaborate very, very
closely with their design, and
marketing, and analytics teams,
and that really paid
off very well, as you
can see in this next slide.
Not only are these
metrics really good,
but there's something
here for every team that
was part of this project.
The product team cares
about the page load time.
The marketing team cared
about organic traffic.
The ads team cared about
the click-through rates.
And the transactions
team cared about how many
buyers and sellers were
able to chat with each other
to complete their transaction.
So these were great
results all in all.
Let's look at a third example.
As you heard earlier today about
Wayfair, the home goods company
here in the US,
that really wanted
to improve user experience
and make sure that they
sustained that improvement.
When they first started off with
the performance budget, which
they said was one of
the most crucial parts
of the measurement
that they did,
it did not take off that much
because these budgets were
understood, but it was not
very clear for everybody.
Well, they made so many
changes towards optimizing
their [? inside, ?]
they said that one
of the most important
things they did
was to socialize their budget.
And the way they did
that was to make it clear
using performance indicators,
as you see here in colors.
So this is how it worked.
They had a budget, and they had
an acceptable variance of plus
or minus 10%.
So if you were within that
range, you're in orange.
If you exceed that
budget by more than 10%,
then you enter the red area.
And if you're better than
the budget by more than 10%,
then you've hit your goal.
Congratulations, you're a green.
They also had a competitor
line, just to make things fun.
This is how it looked
for all internal users
when they looked at
their specific pages
since the budgets were
set at a page-type level
for every route in the site.
So the moment you look at your
mobile site within the company,
you were able to see
these indicators that
told you whether you are in
the red, green, or orange.
As you can see
here, the way they
set these budgets was the moment
you hit the goal reliably,
your budget, and
therefore, goal,
which was always 10%
better than the budget,
tightened up a little bit.
You can see that over here,
where once you hit the goal,
the bar was raised, so
to speak, and it actually
tightened up over there.
By the way, isn't
this the kind of graph
we all want to see, going from
2.3 seconds to 0.9 seconds?
That's really awesome.
This is a performance portal.
As you can see, in
this portal, they
had a clear set
of shared metrics.
Not just the metrics, but
also the page level indicator,
and they have a call to
action at the bottom for users
to basically reach
out to a Slack channel
if they had questions
about these metrics.
Whether you're trying to use
a performance budget idea
or trying to use the general
idea of a shared metrics
automated tool and
actionable next steps,
thinking about your
measurement strategy
at the beginning
of your project is
crucial to the success
of the project.
Another thing to remember
about performance budgets
is that it needs
governance to think
about the metrics at
a page or route level,
and also to maintain those
budgets and metrics as moving
targets as you go forward.
It also needs to be
enforced in a way
that everybody can come together
to meet or beat those budgets.
And that makes it
very, very successful.
CHENEY TSAI: We would
love to go deeper
into some of these partner
stories, but true to form,
we might be slightly above our
performance budget for time.
So hopefully you learned
something from today's talk.
I think we want to focus on
making sure your tools actually
improve the user journey.
And along the way, you might
run into these three problems,
org alignment, technical
approach, measurement.
This isn't anything
new to us, and we
hope some of these
partner examples
help you move a little
further along on that journey.
RAMYA RAGHAVAN:
Which means you need
to do something differently
the next time you work
on a performance project
to build consensus
and visibility for
your project, to cross
those technical hurdles to
reach the North Star Goal,
and also, not just
to prove the impact,
but also to sustain
the improvement
that you've worked so hard for.
When you have all of
these considerations
for your project, I'm sure that
working on performance projects
can be very exciting
and rewarding,
as many of these
partners would attest to.
I'd like to say a special thanks
for all of these partners that
allowed us to share their
stories with you today
and many others who've
influenced this talk
in some way or another.
We would love to
hear more from you
and your specific
challenges when
you're trying to adopt
modern web technologies.
Please come find us at
the [INAUDIBLE] section
during the break.
Thank you.
CHENEY TSAI: Thanks, everybody.
[MUSIC PLAYING]
