[MUSIC PLAYING]
ELIZABETH SWEENY:
So I am Elizabeth,
and I'm a product manager on
the Chrome web platform team.
PAUL IRISH: I'm Paul.
I'm a software engineer.
So we didn't really know
what to talk about up here.
ELIZABETH SWEENY: Yeah,
so when in doubt--
PAUL IRISH: --we
just ran Lighthouse.
Yeah, ta-da, gave us an agenda.
ELIZABETH SWEENY: What
an all-in-one tool.
PAUL IRISH: Yeah.
ELIZABETH SWEENY:
So at I/O this year,
Paul and I laid out a
comprehensive blueprint
for your performance success.
PAUL IRISH: So it had
15 steps, everything
that you need to successfully
measure, optimize, and monitor
the performance of your sites.
ELIZABETH SWEENY:
Yeah, and today we're
really excited to take a
subset of that blueprint
and dive a bit deeper.
So we're going to use web.dev as
our example throughout the talk
today.
PAUL IRISH: Cool,
let's get started.
ELIZABETH SWEENY: Cool.
PAUL IRISH: So the
first thing that we're
going to do is take a
few minutes to measure
and benchmark quality, sharing
some of the newest metrics
that we've been
working on and updates
coming into the Lighthouse
performance score.
ELIZABETH SWEENY:
Yeah, then we're
going to jump right into how
to use our tools for production
monitoring and how to
diagnose and fix problems.
PAUL IRISH: We're not going
to forget Lighthouse plugins
either.
ELIZABETH SWEENY: No.
And we'll finally talk about
our newest addition to you,
Lighthouse tooling
family, Lighthouse CI.
PAUL IRISH: OK.
ELIZABETH SWEENY: So
let's get started.
Measuring the quality
of a user's experience
is not always obvious.
And we know you've
heard it before,
but some things never
change, and that's the fact
that there are two really
critical elements to measuring
the performance of your page.
So lab data, which
is synthetically
collected in a reproducible
testing environment,
is critical for tracking down
bugs and diagnosing issues
before your users see them.
But field data allows
you to understand
what real-world users are
actually experiencing,
conditions that are kind
of impossible to simulate
in the real lab.
The real world is messy with
permutations of devices,
networks, cache conditions.
The list goes on and on.
Either set of metrics
taken in isolation
aren't nearly as
powerful as when they're
taken together and combined.
PAUL IRISH: So you have easy
access to both lab and field
data in tools like Lighthouse,
PageSpeed Insights, Chrome UX
Report, Search Console.
So these tools allow for
local measurement as well as
production monitoring,
which we're going
to get more into in a bit.
ELIZABETH SWEENY: One
thing I wanted to address
was why metrics change.
As we know, there are a
lot of different aspects
of performance that can
frustrate or delight a user--
you know, waiting, uncertainty,
hipster pineapples.
OK, hipster
pineapples are great,
but things that take them
away from me are not great.
So without having a clear
way to accurately measure
the nuances of
users' experience,
it isn't really possible to
quantify the impact that they
have on your bottom line
or track improvements
and regressions.
That means we need to
have our metrics evolve
as our understanding and ability
to measure user experience
develops.
Here is where our
metrics have been.
Across lab and
field, we're trying
to capture load
holistically as well
as the interactivity of a page.
But there are gaps.
And there are opportunities
to more accurately
and comprehensively represent
a user's experience.
So you can remember when we used
to focus on metrics like DOM
content loaded or on load.
They were measuring very
technical aspects of a page,
like when a DOM tree
was fully constructed.
And they don't necessarily
correlate to what
a user actually experiences.
And as we evolve
our metrics story,
we want to fill in as many
gaps of our measurement of user
experience as we can.
There are a few major shifts
here that I'd like to call out.
The first one is the reduced
emphasis on First Meaningful
Paint and First CPU Idle
We're excited to have metrics
available that better capture
when a page feels usable, which
leads me to the second
major call-out, which is
the new metrics you see added.
Now at I/O this
year, Paul and I did
share that they were--
these metrics were coming,
and they were on their way.
But now they're here.
So Largest Contentful
Paint, Total Blocking Time,
and Cumulative Layout Shift
are all out and available.
These new metrics
help to fill some
of the gaps in our
measurement story.
For instance, when
time to interactive
does a good job of identifying
when the main thread calms down
later in load,
total blocking time
aims to quantify how
strained the main thread is
throughout the load.
So they complement each other
and provide that balance.
So let's explore the
new metrics themselves.
Largest Contentful Paint aims
to measure exactly what its name
sounds like, what
is the main largest
content visible for the user?
When is it?
Metrics like First
Contentful Paint are helpful,
but they only capture
the very beginning
of a loading experience.
And LCP helps to flesh
out how quickly a user is
able to see the meat of what
they expect and want on a page.
So yay, I expect to
see my dog, Annie.
It's actually my dog.
And so as soon as I can see
her entire cute, little face,
that's the value of my LCP.
Then there's this.
It's def as not as
cute as my dog, is it?
PAUL IRISH: Nope.
ELIZABETH SWEENY:
No, not as cute.
Just a few pixels moving
can be the difference
between happiness
and complete mayhem.
So Cumulative Layout Shift
measures an aspect of users'--
of a user experience that we're
excited to build out over time.
In addition to our load
and interactivity metrics.
And that's predictability.
Cumulative Layout Shift
measures the amount
that the elements
within the viewport move
around during load CLS is
trying to measure things
like prepending and add to a
list in a way that increases
the chances of accidental
clicks like this one,
or having a new page in which
the story's photos shifts down
the page when it fully loads,
all that kind of stuff.
PAUL IRISH: All right,
so that brings us
to Total Blocking Time.
I'm going to take
a second on this.
And we need to do
a quick refresher
on how event handling
happens in the browser.
So let's say we
have a main thread,
and then the user taps
on a button, right?
So we have some input.
Now JavaScript has
to react to that.
So we have an event handler,
touch start, click handler,
whatever, some JavaScript.
Then we're going to have
recalculate styles, layout,
paint.
We ship a frame.
So that's just kind of
the basic, normal flow.
But let's say that we
take a main thread,
and there's something
happening on it already,
just some JavaScript
that's doing things.
And then, but the user
still clicks on that button
in the middle of that task.
Well, in that case, we
have to kind of wait a bit.
And then we can do the event
handling and ship the frame.
But so there's this
delay here, this input
delay where we're waiting.
Of course, if it's just a
little bit of input delay,
the user shouldn't even notice.
And a little bit would probably
be about 50 milliseconds.
Up to that, user won't
even pay attention.
But if it's over that, the user
is going to start to notice.
So what we're
interested in here is
kind of quantifying that risk
of how much this long task might
potentially impact any
sort of user interaction
that could come
along at any time.
So if we zoom in--
and let's take this task.
So it's 150 milliseconds long.
So basically, the
first 50 milliseconds,
it's going to freeze.
Because if all your tasks
are under 50 milliseconds,
you're good.
No problem.
But once we get above
that, it's a problem.
So the 100 milliseconds
at the end,
we call this blocking time.
So Total Blocking
Time, basically,
if we take a main thread
that looks like this,
we have 270 milliseconds, in
total, of tasks on main thread.
And 60 milliseconds
of that surpasses
that 50-millisecond overage.
So that's our Total
Blocking Time.
So the TBT is just the
blocking time summed up,
all of it between the
First Contentful Paint
and the Time To Interactive.
That's TBT.
OK, so we covered a few metrics.
They're available
in various places.
Pretty much all the lab metrics
are available in Lighthouse.
First Input Delay only really
makes sense in the field.
So then when it comes
to field metrics,
you can see most of them are
available in Chrome UX Report.
[INAUDIBLE] them, you
can collect yourself
with the Performance Observer.
And you can check out, actually,
the documentation for all this
in web.dev/metrics,
brand-new landing page,
which summarizes all these
metrics and also how to measure
them yourself.
Cool.
ELIZABETH SWEENY: So I know
what you're all thinking--
OK, this is all fine and great.
But how do I make my
Lighthouse score better?
So ultimately, optimizing for
the metrics we just went over
will allow you to move
your scores from stuff
like this to this.
So let's take a
minute to make sure
that we know exactly how
the Lighthouse performance
score is actually calculated.
We're going to be updating
the Lighthouse performance
score soon.
We're still working to ensure
that all the scoring curves are
finessed and our
metrics are mature.
But we're going to ship
roughly in January.
This is what the current
v5 score looks like.
It has five metrics that are
weighted and blended together
to form the 0 to 100
performance score that you
see then populated in the
gauge at the top of the report.
The new v6 score that
we're working on now
replaces First CPU Idle
and First Meaningful
Paint with Largest Contentful
Paint and Total Blocking Time.
So let's take a
look at that change.
PAUL IRISH: I got
a little animation.
We got to pay
attention right here.
Ready?
All right.
Take those ones away,
bring in the new ones--
ELIZABETH SWEENY:
It's so beautiful.
PAUL IRISH: --and
then re-weight.
ELIZABETH SWEENY:
It's so beautiful.
PAUL IRISH: OK, so that's
the change we're making.
ELIZABETH SWEENY:
It took us hours.
You can see that we aren't just
updating the metrics to reflect
TBT and LCP, but we're
also adjusting the weights
to be more balanced
between phases of load
as well as
interactivity measures.
Cumulative Layout Shift
is still being finessed,
but we plan to
incorporate it eventually.
So let's jump right
into some of the tactics
to actually improve
these metrics.
Just a quick, cool new feature
in the interest of allowing you
to measure the user's experience
as accurately as possible,
we've added a reanalyzed
prompt to PageSpeed Insights.
PAUL IRISH: So for sites that
are redirected to a new URL,
you're prompted to
rerun the report
on that post-redirect URL.
So you get a full,
accurate picture
of, OK, from the redirect,
what is the story of that page.
So you can go to the PageSpeed
Insights report to benchmark.
It's a great place to start.
But we also want
to know how things
work over a
historical perspective
and see what the trends are.
So the PageSpeed
Insights API itself,
that is underlying the web app.
And it's free to
use and allows you
to scale up your monitoring
across multiple pages
and domains.
And the cool thing is you can
just hook it up to a Cron job,
ping it daily, drop the
results the Google Sheets,
and chart it.
And I actually did
that here with web.dev.
Was able just to do that, a
lot of conditional formatting.
Love conditional formatting.
And then got these
charts, right.
And so this is actually
like two weeks of data
from web.dev homepage collected
by PageSpeed Insights,
the homepage, and then also one
of the articles that I found.
So interestingly,
First Contentful Paint
is about the same in both cases.
But then the speed
index and TTI are quite
a bit higher on the article.
Like, the homepage
is fast-- cool.
But the article--
ELIZABETH SWEENY:
We should probably
investigate that at some point.
PAUL IRISH: Yeah,
we'll go back to this.
ELIZABETH SWEENY: OK, cool.
So it's exciting to see
what our partners have
been able to do with the
production monitoring
capabilities that
the PSI API offers.
This is a snapshot
of a dashboard
that VRBO created with the API.
And they're able to track their
long-term performance trends
and ensure that their page
speed performance remains on par
or better within
the travel industry.
Another way to track
and improve your metrics
is with Search Console's
new speed report.
It was just formally
announced lock last week,
which is kind of cool.
The speed report
allows you to get
a sense of your overall
status of the site
and is powered by data collected
from the Chrome User Experience
Report.
You're able to then go
to PageSpeed Insights
to get the actionable
recommendations that are
surfaced in the speed report.
And we're making it rain
with origin coverage.
OK, yeah, it's a
stretch, but I had
to find a reason to use this.
Because, oh my god, it's a
plug, and money, and pink.
Come on.
But, OK, in all
seriousness, every time
Paul and I get up
here and do a talk,
the CrUX team has done an
amazing job and expanded
coverage to even more origins.
So typically, about a
million jump every time.
That's really exciting.
It does provide us
with the ability
to better understand the current
state of the web in a really
comprehensive way.
PAUL IRISH: And
as of this month,
we made the change the
Lighthouse Chrome extension.
So it will no longer run
locally on your machine.
Instead, you'll
just hit a button.
And it's going to go ping
the API PageSpeed Insights
and run on our
infrastructure instead.
If, of course, you're
testing something
on localhost or something
that's not publicly accessible,
then just open up DevTools
and run it in the Audits panel
instead.
I wanted to briefly touch on the
reproducibility of Lighthouse
numbers.
So you might have noticed
that they change sometimes.
And we see this as
two separate concerns.
So first, running
Lighthouse for the same page
in different
environments-- say, like,
DevTools, PageSpeed Insights,
Lighthouse on web page test,
you might end up with
different results.
So we are investigating
a calibration method
that might be able to
mitigate this somewhat.
But in general, I'd
say it's fair to expect
that different conditions
and hardware will
lead to different
measurements for now.
So variability,
on the other hand,
is the issue when
you see numbers,
and they're moving
around even though you're
on the exact same environment.
So this is a hard problem
that all engineering
teams that measure
performance struggle with.
We're always working to mitigate
variability when we can.
And while we have
more work to do,
we're sharing some documentation
on the sources of variability
and suggestions with what your
team can do to implement them.
OK, so now we're on to how
Lighthouse can be expanded
and extended as a platform.
So we want to make sure
that Lighthouse is tailored
and actionable for you.
So we're making a few--
going to take a few minutes to
talk about how you can leverage
it for your own use cases.
ELIZABETH SWEENY: So
there are two primary ways
that you can build
on top of Lighthouse
depending on what
your goals are.
The first is with stack packs.
PAUL IRISH: There we go.
ELIZABETH SWEENY: Thank you.
Stack packs is a
feature that allows
for stack-specific
advice to be served
within the report itself.
So Lighthouse automatically
detects whether or not
you're on a CMS or
using a framework.
And then it will provide
you with guidance
that is specific to your stack.
These stack packs take the core
audits of the Lighthouse report
and layer customized
recommendations on top of them
that are curated by community
experts that know the CMSs
and frameworks.
We're excited to be launching
stack packs with frameworks
and CMSs like Angular,
WordPress, Magento, React,
and AMP.
We're actively
wanting to encourage
you to create a stack pack.
So please do check out the
repo and work with us on it.
PAUL IRISH: So the second way
that Lighthouse can be extended
is plugins.
So Elizabeth and I announced
them earlier this year
at Google I/O. We're
excited to see the community
start to build their own.
ELIZABETH SWEENY: Yeah,
so let's just take
a second to kind of
refresh our memory
on the impetus behind plugins.
There's a lot of quality checks
that aren't currently covered
by Lighthouse core
audits, and that's
for a few different reasons.
One is that we want
core Lighthouse audits
to make sure that they're
applicable to all web
developers.
There may be really great audits
for an e-commerce checkout
page, for instance,
patterns that you definitely
want to follow.
But those checks wouldn't
be relevant for everybody.
There are also a lot of
really valuable audits
that are relevant to
all web developers,
but the Lighthouse team
just hasn't gotten around
to doing them yet.
So this is where
Lighthouse plugins come in.
They allow you to implement
a new set of checks
that will be run by
Lighthouse and added
to the report as an
entirely new category.
Whenever Lighthouse
loads a page,
it collects a bunch
of great stuff,
and all sorts of data about
its elements, network records,
and just so much more.
It's a complete wealth.
So these artifacts can
be leveraged for plugins.
Once you know what data
is available to you,
you can think through what your
goal is for creating a plugin
and how actionable,
consistent, and valuable it
would be for a user
to be presented
with that information.
PAUL IRISH: OK,
let's make a plugin.
ELIZABETH SWEENY: We
saw something earlier.
PAUL IRISH: Yes,
right, that page,
that article page on
web.dev that was quite slow.
I looked into-- we looked
into why it was slow.
You know what it was?
It was a YouTube embed.
We were like--
I do like YouTube.
ELIZABETH SWEENY: It's great
PAUL IRISH: I watch YouTube.
It's good.
But you know what, those
little iframe embeds--
pff.
ELIZABETH SWEENY:
Let's show them.
PAUL IRISH: Yeah.
This is my desktop machine.
This is a very powerful machine.
And it took half a
second of main thread.
The main thread
was just taken up,
doing things just as
the iframe was loading,
just to show me the thumbnail
and the red play button.
ELIZABETH SWEENY: Did
you try it on mobile too?
PAUL IRISH: I didn't.
I didn't.
I'm scared.
It would be a lot more.
So that's not the best.
There's a few people that
have looked into this too.
And the quick hack is
essentially, well, you just
take the thumbnail image and put
a red play button on it, just
ones that you created.
And then when the
user clicks on it,
just throw the iframe in
there and have it AutoPlay--
kind of straightforward.
So anyways, I made a stupid
little custom element
that does the exact same
thing, really straightforward.
And the nice thing is it takes
500 milliseconds of main thread
time down to zero and, what is
it, 600k of network activity
down to 1--
so a nice overall win.
So let's turn--
let's write a plugin
that just identifies these
normal costly iframe embeds.
So it's basically we
start out with this.
We just name it.
And the only important thing
here is required artifacts.
So we're just saying,
OK, Lighthouse,
we want the data
about iframe elements.
And we're going
to work with that.
So the actual
implementation's here.
We take that iframe
element's artifact,
and we just look it over.
And basically,
we're just looking
to see where it's
coming from YouTube.
That's roughly it.
And we're essentially
going to fail
the audit if we find iframes.
Otherwise we'll pass
it, because we're good.
In a plugin, you can then decide
how you want to score things.
Because it's your own category.
You have a 0 to 100 score.
You can decide how
things are weighted.
You have a lot of control there.
We did this, packaged it up.
It's a really straightforward
and small little plugin.
And you get a nice-- this is
a failure state, of course.
But easy enough to just
identify, OK, yeah,
there's a YouTube
embed here on the page.
It's not optimized.
We should probably
go implement it.
So I ran it against
web.dev, found the failure.
And then I was like,
well, let's just
apply my little custom
element hack to the page
and make it faster--
not 400 times,
I mean, but faster probably.
ELIZABETH SWEENY:
Just marginally.
PAUL IRISH: And that
works really well.
I feel confident about it
being a much better thing
for the page.
But at the same time--
ELIZABETH SWEENY: I mean,
yes, ditto, I trust you too.
PAUL IRISH: Thank you.
ELIZABETH SWEENY:
But we need to make
sure that the plugin and
the light YouTube embed is--
that the fix is viable
and sustainable,
and we can show
that to our teams.
PAUL IRISH: So--
ELIZABETH SWEENY: And so that's
when Lighthouse CI comes in.
PAUL IRISH: Yeah.
ELIZABETH SWEENY: So you can now
run Lighthouse on every commit
before you push to production.
PAUL IRISH: Yeah.
And so we really want
to be able to provide
that kind of confidence
for the entire team
and that transparency.
So for every single pull
request, what sort of change
are we looking at here?
So a brief architectural
look-- so inside your CI
where you're doing your
unit tests and your linting,
you can run the
collection agent.
So essentially, it's just
going to run Lighthouse
a few times for each URL
that you're interested in.
Maybe you're going
to search some things
against a performance budget.
You can then update GitHub and
then post the data to a server.
So Lighthouse CI server
stands on its own
and has a nice front
end that we're going
to take a closer look at now.
So the setup is basically like,
if this is your travis.yml,
then you're building your site.
You're running your own tests.
And the last line is basically,
let's run Lighthouse CI as well
on a couple URLs.
You can set it up just
on your own like this.
There's also an option
of using a new GitHub
action, a Lighthouse CI action.
We've been working on this
with our friends at Treo.
And it kind of batches it up and
runs it inside of GitHub's CI
machines as well.
ELIZABETH SWEENY: And so you're
able to see your entire project
overview, how your
branches have performed
over time against master.
And the trends for your
Lighthouse categories
are all there and visible
to the entire team.
PAUL IRISH: And
so the integration
with GitHub, you'll
see on a pull request,
is in the little commit
statuses pointing
to every URL that's tested
and what the results are.
ELIZABETH SWEENY: And
thank goodness, gone
are the days where you're
having to toggle and compare
between two Lighthouse
reports side by side
trying to find the differences.
And you're actually able
to see the diff view.
Now you can see the changes
between the two reports
in a single view.
Now not only are the
differences in the performance
metrics visible, but also--
PAUL IRISH: Yeah, so actually,
this right here, this diff
is the diff of the page, the
article page that I measured
that was very slow before.
Has the YouTube embed.
ELIZABETH SWEENY: Plus 20.
PAUL IRISH: Yeah, that's a
plus 20 performance score.
I felt good about that.
That was not so bad.
Yeah, so this, I will admit,
it has the YouTube embed fix
and the image optimization, our
sponsored image optimizations
from the keynote--
so both those two.
And so that's good.
So performance score went up.
We can see what happened.
All of our metrics
dropped, which is great.
We can scroll down, see the
summary of everything else that
changed.
So if anything stayed the same
between master and this pull
request, it's just not shown.
It's only presented
with what's useful.
And what's nice
here is, also, you
can just click in
on a few of these.
So for instance, total
number of requests and bytes
for everything-- well,
we see that, images,
we have 1.6 megs less
images on the page--
fine.
I think that's fine.
Main thread work--
our script execution,
we're doing, what, 2 seconds
less, a second and a half
less of work on the main thread.
And also third-party, you
can see the YouTube line
where we drop 600k of
YouTube's bytes from the page.
So in general, feel
good about that.
I'm all right with that.
ELIZABETH SWEENY: Yeah,
that feels like a win.
So how excited is the
team, actually, about this?
PAUL IRISH: We're pumped.
ELIZABETH SWEENY: Yeah,
we're really excited.
So we are launching the 0.3
alpha of Lighthouse CI today.
And you can check it out
in the repo, set it up.
Please let us know
what you think of it.
It is early stages.
And we definitely
want your feedback.
PAUL IRISH: Yeah.
ELIZABETH SWEENY: And now
we're to key takeaways.
PAUL IRISH: Yeah, we're
at the end, all right.
ELIZABETH SWEENY:
So first, we want
to make sure that everybody's
benchmarking with new metrics,
the ones that we
discussed today.
We also want you
to take advantage
of Lighthouse extensibility
with stack packs and plugins.
PAUL IRISH: Yeah, try
out Lighthouse CI.
Give it a shot.
Tell us what you think.
And keep an eye out for the
changes with Lighthouse v6
that are coming out.
We're going to be making
changes with the perf score,
introducing new metrics,
deprecating some older metrics.
So keep an eye out.
V6 should be
finalized by January.
But we'll have a
alpha release of it
for you to try for the
node module pretty soon.
That'll do it.
Here's all the links that
we were showing before.
And thank you guys very much.
ELIZABETH SWEENY: Thanks.
[APPLAUSE]
[MUSIC PLAYING]
