SPEAKER: Testing testing 
testing, this is a captioning 
test.  Nicolas Geoffray
welcome back Chet Haase!
[ Applause ]. 
SPEAKER: Morning, from the very 
much less than full room.  I 
assume that people had fun at 
the party last night, and that 
those people are not here.  
Welcome back to the conference. 
So favorite things you have seen
so far?
Shout it
out.
I owe you $10.  We will make 
that $5.  Motion layout, low 
latency audio, the coding of C++
in the quick talk, it was braver
brave and worked out well.
And he was the DJ last night, 
too.
Anything else? Custom linting,
all right.
GSI, generic system images, 
lightning talks.  Did it work? 
Did people have a problem with 
it, no? I think the yeses win.  
Sorry, we're going to ignore 
you.  What are you looking 
forward to today?
Well, we solved that one.  You 
are beginning to regret that.  
Anything else? Vitals?
Text, firesides, that will be 
interesting.
All right.
Well, the only change that know 
-- I know about the schedule, 
there was an awkward thing with 
the talk that was supposed to 
happen about now.  Jake had to 
go home, apparently babies, 
unlike software, occasionally 
shift early.
[ Laughter ].
So he made a quick departure 
last night on to a plane and 
went home.  We have a different 
talk for you this morning on the
schedule.  Hopefully you noticed
that, if you really wanted to 
see Jake, I'm going to 
disappoint you and you are 
welcome to see the other talk 
about other screen form factors 
in the other room.  Otherwise, 
let me bring on the next 
speakers for the next
talk.
Romain Guy. 
SPEAKER: Did you call me a 
disappointment? We are part of 
the content section committee 
for this event, and somehow we 
ended up with no talk to give, 
so we had to do something about 
it. 
SPEAKER: Unbelievably 
complicated to arrange this.  
His wife was in on it and it 
worked out.  
SPEAKER: Now we are here. 
SPEAKER: And infinity we will 
give a talk and there
is a team and engineers at the 
team that know all about R8 and 
D8, you have the opportunity to 
talk about it in the Android 
studio office hours, Madison in 
particular, and JVG.  So check 
them out, sorry, Jeffrey van 
Gough in the studio area.  Ware 
we're going to talk about 
something completely different. 
We're going to talk about trash.
Why are we going to talk about 
trash? Okay, so there's a common
phrase in software, garbage 
in/garbage out, but nobody ever 
says how fast.  We're going to 
talk about that today.  And why?
Back at I/O, we gave a talk, 
modern Android development, I 
have to apologize for the 
glasses, the sun was right 
there, annoying, couldn't see a 
thing.  The glasses didn't help 
and it did not make us look any 
better.
So Android had certain 
development practices way back 
then, and Android and devices 
and ecosystems have changed and 
we are recommending different 
practices, all the stuff you 
knew about Android development 
may not necessarily be true 
today.  One of the things we 
talked about in particular is a 
lot about memory and garbage 
collection, we are 
recommendations there.  So for 
example, we said back in the 
Dalvik days, it was optimized 
for size and meant to fit in a 
small area, didn't have a lot of
area to do things like  like 
AOT, the optimizations were not 
optimal, the allocations CLEBZs
collections were expensive, 
causing jank, heat fragmentation
was a problem and the 
recommendation was not to 
allocate anything ever if you 
could possibly help it.
And use primitive types 
everywhere, because objects are 
expensive because you are 
allocating them, avoid auto 
boxing. 
SPEAKER: I need to correct you. 
You said avoid allocations 
whenever possible? That's the 
whole point.  
SPEAKER: Yeah, but but
but... 
but they take up space.  That is
1K to 2K? Anyway, it is 
memory-related.
So the recommendation instead 
was to pay attention to ART, it 
turns out that ART is optimized 
for performance and getting 
faster with every release 
because the platform was built 
to opt miles -- optimize more 
and more.  We do AOT, find the 
hotspots, they can run faster 
next time you go through the 
loop.  Applications and 
collections are faster, we will 
go into detail about this.  We 
can defragment the heat and com 
PAECKT -- compact as we go, and 
there's a larger object heat 
that makes the allocations 
faster than they used to be.  So
the recommendations are to 
allocate, it is not that big of 
a deal, be concerned for inner 
loop situations and that you are
actually causing the device to 
do stuff, you are causing 
battery and CPU usage.  So you 
want to be aware of these 
things, but maybe they are not 
such a big deal that they should
affect your APIs and your 
development patterns the way 
they used to.
  However, that was a lot of 
terse information stuffed into a
short amount of time.  We 
thought, maybe we should do this
talk to explain ourselves a 
little bit completely and say 
why is this the case, what has 
ART done to make life better? 
The original idea of the talk is
we said this stuff, but would it
be nice to write dem tow o 
applications and write bench 
marks on this is why and these 
are the canonical results that 
show the premise.  That is hard,
garbage collection in ART is 
concurrent, there is stuff 
happening in the background all 
the time.  If you want to 
trigger that thing right now, 
you will not be able to.  So we 
made an attempt to write some 
demo applications, you will see 
the results here.  But we don't 
have enough canonical 
deterministic data to show you. 
Instead, we will show you the 
background of why it is 
difficult to write these things 
because everything is happening 
magically for you in the 
background.
  First, we will talk about 
memory.  We see a couple simple 
lines of code here.  We have a 
primitive type, Fu, set it to 
five, we will allocate an object
here.  There are different kinds
of memory in the system and 
different implications.  If we 
have a primitive type, it may 
show up in the stack or 
registers.  Dalvik was a 
register-based allocation 
system, so it actually popped it
in over there. And whether it 
shows up in the stack or the 
register, you can think of it as
free.  It is allocated at 
compiler time, it says, when I 
run this line of code, here is 
where I'm going to stick this 
thing.  It didn't need to find 
space, it knows that it has 
space on the stack.  So you can 
think of that as limited, sorry,
limitness storage space, as well
as the registers.  It will find 
a cubbyhole to stash that thing.
It is free.  And however, when 
you allocate any other, sort of,
thing, when you say new object, 
then it needs to find space for 
it in the heap.  That means it 
needs to figure out where it is 
going to fit among the other 
things that already occupy the 
heap and will find space down 
there.  So that's the basic 
memory system of runtime and the
computer overall.
So the idea of garbage 
collection, well, we've all been
doing garbage collection even 
before higher level languages, 
like Java and Kotlin.  If you 
are writing C++ code, you can do
an allocation like this, you can
write the code that uses the 
object you allocated.  If you 
don't do anything else, you just
leaked an object.  You are 
taking up space in memory 
somewhere that eventually is 
going to cause you to run out of
memory.  Right? So what you need
to do is to free the object.  So
you delete the thing and reclaim
the memory, manual garbage 
collection here.  Sometimes you 
forget that you free it over 
here and in this place you 
continue using that object and 
then you crash, maybe.  So very 
non-deterministic system, you 
are responsible for managing 
your own garbage, your own 
allocations and collectionsism .
Tends
to be tedious and error-prone.  
We have languages
like Java, you write the 
language that uses that and it 
is freed.  If you continue using
it, you have a reference to this
object and that means it is 
freed without freeing it too 
soon.  The system knows what is 
going on, you don't have to 
manage this thing and scientist 
it is doing it for you.  If it 
is doing it for you, there are 
questionings s that occur, and 
there is no crash.  There are 
things that occur to you, how 
long does it take for the system
to do this? I know how long the 
statement was going to take in 
C++, how long is it going to 
take for this system to 
automatically find space in the 
heap for me? How long does it 
take to walk the heap, find the 
spot, put this thing, and 
allocate the space? How long duz
it take to allocate the objects,
when the reference goes away, 
how long does it take to collect
these things.  And what impact 
is there system-wide? When we 
are returning this garbage 
collector thread, this heavy 
thing in the background, what 
impact does that have on Jenk on
the UI thread or whatever.
And when do these things 
actually happen?
And also, how efficient is that 
heap usage?
Right, it is not just going to 
malwick theseing CHUNCHG chunks 
for the thing, it is allocating 
space and sorting around.  On 
Dalvik, it is fragmenting the 
heap over time, you have a lot 
of memory available, but you 
cannot access it anymore.  So 
how efficient is that and how 
much memory are you taking in 
the system to do all of this 
stuff? 
SPEAKER: We will start by 
looking at how Dalvik collects 
garbage.  It was the one thing 
that we were using until Android
KitKat and was replaced with 
ART.  So this is a picture of 
the heap, everything in white is
allocated, we have a few holes, 
we are trying to allocate this 
blue object.  Dalvik will walk 
through the heap, find a spot 
where it fits, and once it finds
one, it will slide it there.  
Things are more complicated when
it is time to collect objects.  
So there are four phases, the 
first one, Dalvik has to pause 
all the threads in the 
application to find the root 
set.  So root sets are local 
variables, threads, studied 
variables.  They are just the 
roots of all the allocations 
that can be reached in the 
application.  That takeakes a 
lot of time and your application
cannot do anything.  And the 
second is concurrent, the app is
running again, and from the 
roots, Dalvik will find the 
objects that can be reached and 
will mark them as such.  And 
unfortunately, since the second 
phase is concurrent, the 
allocations can be triggered 
during that time.  So for this 
instance, we are allocating 
another object.  And so we need 
to pause the application again 
and find the reachable objects 
again, all of the threads are 
parsed.
And so finally, we can knock all
of the objects that are not 
reachable and they are 
candidates for garbage 
collection and they are put into
the heap.  If we want to 
allocate something in the heap 
and the heap is pretty much full
and we have an object that is 
marked for allocation, Dalvik 
will go through the heap, 
realize there is no memory 
available, and will target the 
file log.  Anytime you see this,
there's a log on the individual 
lot cat, you can see the 
messages.  So run the 
collection, get to this memory, 
and then we can run through the 
typical allocation mechanism.
However, sometimes the heap is 
full.  There are no objects that
can be collected, everyone one 
is reachable, Dalvik cannot find
anything.  And two things can 
happen, the heap will grow or 
the application will crash.  And
it is the error you see in a lot
of applications, especially on 
older devices.  This happens 
when you are allocating large 
objectz.  I remember a lot of 
developers in the early days 
find bugs against bit map 
factory, because auto malware 
happened during the decoding of 
bit maps, and there was a link 
in the factory because bit map 
objects are big and it is hard 
to find space for them.  There 
is no leak in the factory 
whatsoever. 
SPEAKER: So we wrote a simple 
demo application to show off how
some of this stuff with heap's 
fragmentation works.  So in the 
demo, we allocate chunks of 
memory to max heap.  The heap 
starts very small, if you cannot
allocate an object, it is going 
to grow that over and over and 
over until it reaches the max 
possible.  So for this demo, we 
allocate the 1MG chunks, there 
are 400MG free, it will allocate
these chunks until it grows the 
heap.  You get the error, we 
catch the error, the heap is 
full. 
SPEAKER: This is the only 
situation where you do a try 
catch in the ought  out of 
memory error, don't do that in 
the application. 
SPEAKER: Come we  -- and we say 
there is 1MB free, we will 
fragment the heap and free a 
bunch of chunks.  We allocated 
and freed every other one. 
SPEAKER: Go through the 
reference, set it to null, and 
set it to GCU so the memory goes
away. 
SPEAKER: And we get this result,
the fragmented heap size is 
200MB, we have space to allocate
a 2MB object, we have 200MB 
free.  So what can be the 
problem? So we press the button 
and it says nope, we cannot fit 
that in there.  If you look at 
the log, we have depressing 
things here -- 
SPEAKER: The best error message 
in all of computer science. 
SPEAKER: This is beautiful.  It 
says, okay, you have 200MB free 
out of 400 and we're forcing a 
collection so we can make room 
for a 2MB chunk, we are trying 
really hard to do that and we 
ran out of memory.  We cannot 
find space for 2MB out of 200, 
because aapparently Dalvik is 
bad at math.  You cannot find 
them contiguous and they cannot 
compact the heap.  So we cannot 
shove stuff to the side to make 
room for a larger object.
So ART came along in Lollipop, 
it is a platform for 
optimization, it no longer had 
the memory constraints that 
Dalvik did so they can build in 
the fundamentals that they are 
are improving over time.  Out of
the box, much faster allocation 
times and faster run time, the 
ability to do ahead of time 
compilations, we are running 
binary code all the time and not
jitting things to see how we can
speed things up SQULNCHLTHS  -- 
SPEAKER: When we talk about 
faster allocation in the talk, 
it is the time for the run time.
Not the constructors, it has 
nothing to do with your code, it
is only in the run time itself. 
SPEAKER: All right.  So how did 
our allocation work? 
SPEAKER: In ART, we introduced 
the Rust Adock, we replaced it 
for DL, which stands for dud
dley, the person that wrote it. 
SPEAKER: That's what happens 
when you write a nice algorithm,
people name it after you. 
SPEAKER: It is the kind of 
person that makes me feel 
inadequate at -- as an engineer.
So it is the function call in it
the native code.  This is what 
you work on in C++, the 
algorithm we are using.  Dalvik 
is relying on that.  And
the main benefit is we can do
thread aware, and we will look 
at it in detail.  And a lot of 
the tweaks that have been done, 
so small allocations are grouped
together, to reduce 
fragmentation, and we allow 
large allocations on pages, 
typically four KB on modern OS 
and gives you better 
performance.
And the garbage
collector has to collect logs.  
And overall,
allocations are four to five 
times faster.  You can do 
something really really really 
expensive in your constructor 
and you are still five times 
faster than dlmalloc. 
SPEAKER: And the other very 
important thing with ART is the 
ability to deal with large 
objects in a better way.  So you
have this normalize object, it 
will find space for it in the 
heap over here.  And what 
happens if you have this large 
object, we mean an array of 
primitives or string.
And these are the types chosen 
because we can guarantee these 
objects will not have a 
reference to something else.  
They can live on their 
own. 
SPEAKER: 12KB. 
SPEAKER: That's the heuristic 
for now, right now it is 12K, 
the primitive types are string. 
Where would we put this? In 
Dalvik, we put it where exactly?
In this fragmented heap, there 
is no space.  In ART, it is 
simpler.  The
complicated mechanism looks like
this, we shove it like this, we 
malloc a space in there and 
shove it in. 
We allocate a space in there, 
you are part of the heap, and it
is living on its own somewhere. 
Very fast, very easy.
And it is also a moving 
collector, so we can actually 
compact things.
And so we no longer have the 
fragmentation problem.  However,
it does this in the
background. 
So it is more complicated.  My 
understanding was that if your 
application goes into the 
background, then eventually this
very expensive operation, it can
take up to 100MS, can compact 
the heap.  We don't want to run 
it in the fore ground, we are 
going to Jenk your app, we will 
wait until it is in the 
background, the user is doing 
something else, not paying 
attention, we are compacting the
heap for you.  I will show the 
same fragmentation crash error, 
I will show how it crashes on 
KitKat using dav -- Dalvik and 
it will crash on the releases 
until we can do it in the fore 
ground in a later release in O. 
And this was a cool demo, ran it
on L, and it didn't crash.  It 
will defrag in the background, 
but also in the fore ground if 
it has to.  That's what you 
want.  So if you need that 
memory now,p  wouldn't it be 
nice if it doesn't crash? 
SPEAKER: And this is a 
replacement for the GC file log 
from before. 
SPEAKER: And it says you need 
space for this large object, we 
will compact things and then put
it a where we need to.  So on L 
and above, we run the same 
fragmentation
demo, we malloc to the maximum 
heap size, one MB free, and then
we try to find space for the 2MB
block, compact the heap, and it 
puts it in.  
Very simple. 
SPEAKER: So another improvement,
so remember with the Dalvik GC, 
we had two phases, including two
pauses for your application.  
The pauses water  were bad 
because it was not doing 
anything at that time.  And what
is worse, the pauses can be 
expensive.  So on average, some 
of the two pauses were two 
milliseconds.  And even when it 
was 10, it was pretty good.  We 
have done a lot of performance 
work over the year and I have 
seen these pauses work sometimes
over 100 milliseconds.  So no 
matter how good the UI is, it 
will Jenk.  If the user is 
trying to scroll, it is not 
going to work well.  So one of 
the things that ART does, it 
removes one of the pauses.  So 
the first step, the marking the 
root set, finding the roots that
are reachable in the heap St. A 
concurrent phase.  It doesn't 
pause the application anymore.  
And the other pause we have left
is a lot faster.  Instead of 10 
milliseconds, we have 3 mill  
milliseconds.  It will pause, 
and if GCT happens during 
scrolling, you can reach 60FPS 
without Jenk.
What was introduced in ART is 
the concept of the fast 
collection of the young 
generation, keep track of the 
objects that were allocated 
since the last major garbage 
collection, those are temporary 
objects, and we look at them 
first.  If we allocate memory by
looking at the subject first 
because they are short-lived, we
don't go through the heap. 
SPEAKER: This has an important 
implication for Android 
development, all of a sudden, we
made object allocation and 
collection much easier. 
SPEAKER: It is not free, just 
less expensive. 
SPEAKER: Yes. 
SPEAKER: We will introduce the 
heap that we talked about, you 
have less fragmentation.  And 
one of the benefits of that, 
because we don't fragment the 
heap, we don't have to grow the 
heap as much in all the 
processes.  And of course, we 
don't have
the GCFORALLOC and the faster 
run time, and we can reduce the 
GPAC which has nothing to do 
with the garbage collector.
Marshmallow, I was joking 
yesterday that it is a boring 
release, I cannot remember what 
was Marshmallow.  Here it is, 
optimizations. 
SPEAKER: Things got faster.  
Fine-grain details, things got 
faster.
N, things got faster, allocation
in particular, they rewrote all 
the thing, the cool stuff in 
assembly, and that still helps s
in software.  Now we are up to 
10 times faster in allocation 
costs when you compare it to 
Dalvik.
And now we are in Oreo, they 
rewrote the whole thing.  We 
introduced the concurrent heap 
compaction collector, and we can
compact in the fore ground, not 
just when we do the GC for Alloc
in the space, but it is moving 
stuff around and optimizing what
the heap looks like so 
allocations are faster and 
easier over all.  So 
defragmentation in the fore 
ground, you are not resizing the
heap as often because the heap 
is always -- we are culling the 
things out of it.  There are 
fewer GCs FORALLOC because we 
are not in the situation.  So 
what about the applications and 
services in the fore ground? 
System, play, UI.  We cannot 
defragment those until it is in 
a bad situation.  Wouldn't it be
nice to be in the fore ground so
those are optimized? If we can 
do it for them, we are getting 
system-wide savings on the heap 
size for all of those 
applications.  Smaller heaps for
all means less memory.  And the 
entire device had a 30 percent 
savings on memory requirements.
And so we can take a look at how
compaction works.  In general, 
we have these two 56K buckets 
that are assigned per thread.  
That means again, huge savings 
in not having to lock down all 
the threads to do the 
allocations in collections.  A 
thread, if it needs memory, it 
is responsible for itself.
So a thread says, okay, I need 
memory, it is handed one of 
these buckets, it allocated it 
in there.  And over time, the 
thing is defragmented, there's a
heuristic if it is less than 70 
or 75 percent utliization, we 
can collect it and empty it.  So
we see the T1-3 regions here, we
don't have a lot going on here. 
We can take the memory in there,
all of the allocations and shove
them somewhere else in the heap,
empty out the buckets, and that 
allows something that is super 
efficient, thread local bump 
allocatur.  And all we have to 
do is move the pointer.  We 
don't have to walk the heap to 
find where the free space it, we
put the next object in the next 
available space and we know 
where that is according to the 
pointer.  It turns out all of 
this stuff put together, we are 
now 18 times faster than Dalvik 
for these allocations.  So we 
can see how these allocations 
work in practice.  You can see 
all of these little colored 
objects, we are making space for
them in the different threads 
that need these allocations.  In
T1, we zeoed it out and we 
advance the pointer, the pointer
tells us where to put the next 
one and the one after that and 
so on.  Very fast and easy 
compared to what we were doing 
before.  On the graph, you can 
see the comparison of where we 
were at with Dalvik in 
allocation costs, depending on 
in O, with bump pointer and 
assembly allocations
instead.
Where are we going now? It is 
important to note that the young
generation stuff that we talked 
about as being so awesome is 
currently gone. 
SPEAKER: But it is in O stream. 
It will be in the future 
release. 
SPEAKER: So there are trade offs
here, we believe it is better 
overall.  All of the benefits 
you get from the garbage 
collector and O compensates for 
the older not being there.  Look
for those in a future release. 
SPEAKER: And object pools, this 
is a technique that we have 
recommended using in the past, 
we use it ourselves extensively 
inside the platform.  The wisdom
is reusing an object has to be 
faster than allocating and 
collecting them all the time.  
You can see the performance 
graphs, the X axis is the size 
of the object that you are 
creating or using.  And in red, 
it is the time it takes to 
handle the objects with the 
pool, and to allocate and 
collect the objects.  And until 
N, using a pool is basically 
always a win compared to the 
garbage collector.
But with O, we all the 
implements we've done and this 
thread allocator, synchronized 
pools of objects are generally 
slower.
And if you have a pool of object
that is only on one thread, that
thread only, you are actually 
doing the positioning that ART 
does for you in O   You might 
not see those savings.  In O, 
you should have a synchronized 
pool of object and you are 
probably better off without the 
pool. 
SPEAKER: And there's a certain 
memory size of object where 
these graphs cross.  This is in 
a benchmark application that is 
hammering it and maximizing the 
bandwidth.  So the general 
advice is that you really 
shouldn't use that, especially 
the synchronized pool approach, 
because A, it is error-prone and
tedious to manage, and B, it is 
slower in general.  That 
synchronization access tends to 
be slow e er than what we can do
now. 
SPEAKER: The point of O is we 
don't have the lock in anymore 
for the allocations. 
SPEAKER: So, what are the 
recommendations now? Creating 
garbage is okay.
I wouldn't go out of your way to
create garbage, it is still 
taking time.  You are requiring 
us to take time to allocate 
objects as well as later collect
them.  And you are taking up 
battery and CPU and you are 
causing -- 
SPEAKER: And don't do that, 
Chet, you should see his desk.  
It is pretty disgusting. 
SPEAKER: I like to be a counter 
example.  Creating this stuff is
okay, and so is collecting it.  
Use the types and the objects 
that you need F. It makes sense 
for the architecture, the APIs, 
for the libraries for the code, 
use those.  We are not pushing 
everyone to use end to end bit 
flags everywhere.  Instead, go 
ahead and alCAILT icate when you
need to for the use case.  
However, GC is overhead. 
SPEAKER: We will demo that that 
showcases that. 
SPEAKER: And make the right 
choices for you.
  And, in general, the framework
programmers, we still are your 
interloop.  So we still take the
old practices, because why would
we allocate if we didn't need to
if we can make your inner loop 
faster.  So be aware of the 
inner loop situations, 
otherwise, it  do the right 
thing for your code.
  Wrote a simple application to 
showcase the Jenk you can see 
because of allocations and 
collections.
  During the on-draw I would 
call out to a method to run some
stuff.
And, in this case, we're going 
to test auto boxing.  So we have
an array of 100,000 float object
objects, capital F.  It is not 
the primitive float, it is the 
capital F, we are allocating 
these objects on-the-fly.  This 
is what the method looks like 
that runs on every frame, walks 
the length of the array and 
takes a primitive float and 
stuffs it into the array that 
causes an auto box, so these 
tiny allocations goes into the 
array.  A lot of them over time,
because of the auto boxing 
thing, it is going to cause a 
bunch of allocations and we need
to collect them over time as 
well.
So, if we take a look at the 
demo.  We will pop out.
All right.
So we're 
running on K here, run this 
animation, we should take a look
at the log here.
We run the auto box, we are 
coming -- calling out to the 
method.  If you look at the log,
so we are taking allocation 
times of 28, 24, sort of in 
general, high-20s milliseconds 
and we're causing a lot of GC, 
because that is what happens 
when you do this thing.  We can 
pause this, pop over and see 
what it looks like on O, run the
animation here.  We enable the 
log for O.
We do the auto boxing, and zoom 
in on this.  And the allocation 
times are obviously much less 
than they were before, and also 
more importantly, there are no 
GC FOR ALLOCS.  We are not 
triggering the 
Jenk-inducing ALLOC in this 
case.  There's something similar
for bit maps. 
We will look at the KitKat log. 
In this, you are allocating bit 
maps.  Again, we're getting a 
lot of Jenk there.  If we zoom 
in on the log, the allocations 
for these large objects, a 
thousand by a thousand bit map, 
you are taking in 12 or then 
milliseconds each time and you 
are constantly triggering these 
GC FOR ALLOCs because you need 
to make memory for the new one.
If you pop over to the O log, we
will stop this, go over here on 
the animation, do the bit map 
test, and now we have allocation
times of, well, zero, 1.
Because again, all that it is 
doing is an Malloc, shoving it 
in and it is easy to allocate 
and collect when it needs it.
 Stop that.
All right.
This is the bit map test, 
allocating a thousand by a 
thousand bit map.  You can see 
the results that we did.  
SPEAKER: All right.
So this is a demo to stress test
to the garbage collector and see
what overhead we get.  This is a
ray tracer using rendering that 
I -- 
SPEAKER: You can tell it is a 
rendering because it is a 
checker board. 
SPEAKER: That's right.  And I 
brought it over to Android.  I 
will not run you through the 
code, but this is Kotlin, and it
does a lot of allocations.  I 
have a data class, it contains 
three floats, XYZ, those are 
primitives.  Not the capital F 
objects from Java.  I observed 
the hundreds of lines of code.  
It is math on the object, we 
have float equal X, we do 
division and additions.  And the
way these functions are 
implemented might float through 
is immutable.  Every time I add 
or multiply, I create a new 
object.  So these are small 
objects and for every pixel to 
render, we have hundreds of 
thousands or millions of these 
3s.  So we exercise the GC.  We 
create two system images, one on
KitKat, one on O, both running 
on the emulator, running the 
same hardware, on Nexus, with 
the same number of cores and the
host machine.  The difference is
the garbage collector.  And what
I will press next, both demoes 
will run at the same time.  At 
the top is O, bottom is KitKat. 
See if you can spot the 
difference in performance.
It takes 47 seconds for KitKat 
to render the first dial. 
SPEAKER: But it is a good dial. 
SPEAKER: And we are running the 
exact same application, the 
difference is the garbage 
collector.
So on O, rendering one dial 
takes 100 to 500MS, and on K, it
takes 40 to 50 seconds.  So two 
orders of magnitude slower.  If 
we look at the logs on KitKat, 
this is what you see.  A bunch 
of GC file logs, and I grabbed 
logs for 10 seconds' worth of 
computations.  You can see that 
we are constantly stopping all 
the threads and blocking the 
applications all the time.  
Nothing is getting done.
That's a lot of GCs, and you can
see every time the GC takes 3 to
5 milliseconds.
So if we can look at systrace on
O, even though things are 
better, they are not quite 
perfect.
So this is sys trace here, you 
can see what the CPUs are doing.
And from the far, it looks like 
they are busy.  They have few 
worker threads that are are 
computing this 3D scene.  
There's a lot of cases where the
CPUs are not doing anything, 
there are holes in the pipe.
So if we look at the app
itself, we see two interesting 
things.  
We have the threads that are 
doing the work, and from time to
time, the pause, and you can 
probably not read this, but it 
is full suspend check and you 
are not doing any computation.  
And the reason is that we have 
this thread, the heap test 
demon.  It is the concurrent 
garbage collector.  So even 
though we are doing garbage 
collection and allocating so 
many orbital smany objects, it 
takes so much timetime.  It is 
blocking anyway.  
It is not pausing because we 
have to pause them, it is not 
for the algorithms, because the 
GC is still busy.
  And I was running more threads
and running into rough situation
situationssituations, I had so 
many threads that they are 
starting the GC thread.  And as 
a result, we are only getting 
about 30 to 40 percent CPU 
utliization.  We don't have all 
the power on the device.  So 
what you saw on O is three times
faster than that, if it was not 
allocating as much.
And so one of the things we will
talk about, we have four minutes
left.  You have to be careful 
when you create benchmarks, 
because the garbage collector 
can affect the results of your 
benchmark once, well, not really
the benchmark itself, but the 
algorithm you are benchmarking. 
Once it is inside the 
application, it behaves 
differently.  I will skip this. 
With  when you have a CPU, you 
have the gold cores.  And everyb
core has a nil 1 cache, every 2 
core, and there's an L3 cache 
shared by all the cores.  When 
you want to access data, I have 
a float array.  The first time 
we assess the float, the CPU 
looks at the L1 cache, sees if 
it is there, goes to the L 2, if
it is not there then to the L3
and the program.  And it is more
and more expensive as you go up 
the chain.  So the L1 takes a 
new nano seconds, 2 is 4 times 
that amount, 3 is 10 times 
slower, and so on.
And I wrote a demo that 
allocates the list
of flow arrayarrays.  Each is 
16MB, represented by the red 
lines, and I built a benchmark 
using that.  If I look at all of
those arrays of floats, this is 
what it looks like in RAM.  They
are neatly stacked together in 
RAM, and I'm using a width of 64
bytes here for a reason we will 
see in a minute.  And then I 
wrote a simple algorithm, I go 
through the array, I wrote 
computations, we will see what 
happens to memory.  So when we 
access the first float array in 
the list, it is not in the 
cache, it is in the RAM, and/or
L1, or the 2 or 3.
  And they are not going to 
fetch only one byte, but 64 
bytes at a time.  By fetching 
the first array, we fetch the 
next three at the same time.
So when I want to read the 
arrays, nothing happens because 
they are in the L1 cache.
So this is pretty efficient.  
And then we run our computation.
And in the test app, I have the 
initialization of the arrays so 
we can allocate other stuff.  
And I'm doing this to basically 
replicate what happens when the 
GC moves things around or you 
have fragmentation in the app, 
and for any number of reasons 
that you are seen before, they 
are not next to one another in 
RAM.  So I'm representing this 
with a bunch of gray lines.  If 
you run the algorithm again, we 
fetch the first array.  And 
instead of fetching the other 
did I tell  data that we want, 
it is put in the L1.  And in the
next array, it is not in the L1,
we have to go back to memory and
get it.  We are running the same
algorithm, but now we have to do
more.  The CPU has to do more 
work.
And we can re-create the same 
thing by spacing out the arrays 
more so we will not find them in
the L2 or L3 and force the CPU 
to do more work.  If we run the 
different variants of the 
algorithm, all we did is change 
the way we alquate the objects, 
running the same computations.  
When everyone is stored together
in RAM, the algorithm takes 
about 150 milliseconds on the 
pixel thread.  That the no 
thresh.  And when I space out 
the allocations so we cannot 
find the data we want in the L1,
we almost twice as slow and 
running the same computations, 
but the CPU is busy and fetching
data in RAM.  If I space out the
computations more so we find it 
in the L2, it is 5 times slower.
The same exact algorithm.  So if
you write benchmarks, that is 
really good.  You should 
probably do that.  Be very 
careful, be aware of the fact 
that the numbers getting to 
benchmark might be very 
different than the numbers you 
get in the app running on your 
user's devices.
And, yeah, you are actually 
benchmarking the CPU access 
patterns.
And with that, we are done.  We 
have six seconds left. 
SPEAKER: Very important thing, 
if you are interested in what we
talked about today, it there's a
deeper version of this as well 
as the runtime improvements by 
engineers on the ART team.  So 
check that out, it is at the 
same time as the fireside chat 
at 11:10.  Please go and see 
that talk as well.  And that's 
it.  Thank you.  
SPEAKER: 
Thanks. 
SPEAKER: [ Applause ]
.
Coming up next: Use Android Text
Like a Pro by Siyamed Sinir.
GZ
name is Siyamed Sinir, I work on
Android text.  And today, I 
would like to talk about why 
improving your text measurement 
performance is important for 
your overall apps performance.
In order to do this, I wanted to
gather some numbers and, 
therefore, I wrote a simple 
sample app that has a feed and 
is similar to the applications 
we use every day.  It has a list
of items, image, and some text. 
And when we look at the text, as
you can see in the picture, the 
user name and the title for each
feed item is what we call single
style text.  And the content 
contains a text that has 
different colors, text sizes, 
fonts, etc.
  One important point here that 
I would like to emphasize, on 
our platform right now, the 
hyphenation is turned on by 
default.  However, in this app, 
as you can see, there is no 
identificationation applied -- 
identificationation applied, and
Android runs in correct words 
side by side, it cannot do any 
hyphenation.  So when you look 
at each feed item, it has a very
simple structure, a linear layer
at top, the image layer, and 
fields for title and content.  
I'm pretty sure it is the same 
or similar to one of your app's 
existing design or layout 
structure.
And what I wanted to see was, 
for each field item, what is the
percentage of the time spent on 
text versus the whole layout.  
So what I did, I wrote a 
benchmark where I called the 
linear layout measure, on one 
end, and meanwhile I recorded 
all the parameters that were 
passed to the different text 
fields, and I assimilated the 
same thing on those text fields.
Because of the layout here, the 
TextViews at the top were 
getting two calls
each.
In order to have consistent 
numbers or
And just turning the hyphenation
off, and turning off hyphenation
did not have any affect on the 
UI.  Finally, I wanted to check 
how the had beenumbers would 
improve, if I wanted a new API 
that I added on Android P and 
AndroidX, for computed text. 
These numbers were crazy, 4 
milliseconds becomes 20X.  It is
almost a 2X improvement.
And now I want to describe how 
predicted text
works.
I want to show what kind of 
things we did while doing the 
computed text.  We asked 
questions, like why text 
measurement is so expensive, and
since almost at the beginning, 
we did the text performance, and
as you understand from the 
number, 98, it is always going 
to be a bottleneck.  Therefore, 
we tried to see, can we move all
of it and provide an API.  And 
when the system calls measure on
text field, it does the 
preliminary work and identifies 
the width and height of the text
edit it contains and according 
to the parameters that are 
passed to on measure, and the 
numbers that it finds, it 
creates a layout.  All of these 
processes goes into our native 
code.  Because we have to use 
some open source libraries such 
as 
freetype to measure our text.
And 90 percent of our time is 
spent on the native code.
And when we process a string 
like this to our native code, 
what it does is it divides it 
into words, and then for each 
word, it applies text shaping.
It finds the font that can 
render the collector and tries 
to bring those collectcollectors
in the work together.  And then 
we check out much time this 
takes, compared to all the time 
that is spent on the native 
code.  It is again more than 90 
percent.  So text measurement, 
taking the 90 percent of the old
measurement, native code taking 
ninety percent of the text, and 
shaping taking more than 90 
percent of the native code means
that most of the measurement 
time for this app is spent on 
text shaping.  
The system applies to the same 
rules for all of the words, for 
each word that is measured, it 
caches them.
One of the reasons to cache them
is, just after the measure or 
layout, there will be a draw 
operation, and the draw will 
need the
same information.
Then it applies line breaking 
and identif hyphenation.  Some 
one question here is, why does 
hyphenation, turning it off, 
improve the performance twice? 
It is mainly because of, 
whenever it tries to hyphenate, 
it has to apply text shaping for
more words.
Let's say that, in here, it the 
the example is divided into X 
and ample.  It has to do two 
more shaping on two more words.
So the next creation is, if we 
were to move this expansive 
measurement background thread, 
what we would need.  The first 
issue we encountered was, before
P, was acquiring giant 
synchronization locks.  We fixed
it on Android P, we made it  the
locks smaller.  Can we use the 
layout classes for such a goal? 
There were two issues.  Layout 
classes needed a width to be 
provided to them so they can do 
their calculations.  And that 
width, you would not have that 
width before measuring.  So it 
was a chicken and egg problem.
The other one is the layout 
objects are just reprints for 
the text it contains, like a 
cache on the Java site, and they
don't know about the previously 
calculated layout objects, they 
know how many legss layers there
are, the start and end, and the 
coordinates.
So we wondered, what if we 
created the construct? First of 
all, it doesn't need a width.  
Second, it should be able to 
create it on a background
thread safely. And it needs 
strong references to the native 
layout word objects. This is 
important, in terms of layout, 
if you were to create a layout, 
before you need it, then maybe 
when it is time to measure or 
render it, you will use the 
layout objects, they will be in 
the cache and there was no far  
guarantee you would have steady 
improvement.  
We wanted to extend this 
construct, because with  we want
it to be compatible with the 
current APIs and also your 
applications.
Since it is mostly interested in
how the text will look, this 
construct needed some parameters
that will change the next 
styling, such as the text size, 
color, local,
and etc.
  One important thing is, that 
was the reason why previously 
those native objects were being 
cached and edited.  And it is 
the amount of memory that they 
use.
And right now, when we use 
PrecomputedTextPrecomputedText, 
you spend 20KB for 500 
characters.
  Even though it is spanable, 
with a mutable interface, every 
calculation that the text does 
is done at the construction 
type.  You should
not be calling set span or 
remove span functions, with 
styling information that will 
change how the text will look, 
otherwise you will get an 
exception.  That will invalidate
all of the computation and it 
will be useless.
And then we look at the 
parameters that PrecomputedText 
required.  Since most of the 
text styling information is 
right now encapsulated in the 
text pane, it requires the text 
pane as a mandatory constructor 
argument.  The others are break 
strategy and text direction you 
will know from the static layout
builder.
You will be designing the text 
styling in the XML, you want the
helper function where you can 
create the PrecomputedText 
parameters using the text field.
However, at this point, I would 
like to emphasize that TextView 
is not required to create a 
PrecomputedText per am, it is a 
helper function, and it will 
make more sense when we go over 
how to use PrecomputedText with 
RecyclerView.
Since version 25, re
RecyclerView has a prefetch 
feature.  And not only during on
the unbind folder, you will call
set text and the recycle view 
will measure the whole layout 
that you just created.  And we 
know that the measurement part 
of the TextViews are expensive. 
When you use RecyclerView with 
PrecomputedText, you will change
set text to set text
feature, and what 
PrecomputedText will do, before 
the item is shown on the screen,
go to the background thread, do 
the text measurement and it will
show the PrecomputedText.  And 
when the computation is ready, 
it will switch back to the right
thread and it will call set text
with this PrecomputedText that 
makes the measurement part much 
faster than the
previous case.
SPEAKER: You just change it to 
be set text feature and some 
configuration for para computed 
text.  And here, what you tell 
the system, please run this task
on the background thread, and 
para compute my text with this 
sequence, my data, and this 
display configuration
configuration.  And this becomes
very handy to complete the para 
computed text per ams. 
And one point here, if you are 
changing the text styling 
according to the data you have, 
you have all of the spanables 
before calling
this function.
So you can turn off hyphenation 
in your app globally.  This is 
simple, and you saw the affect, 
it increases the measurement 
performance by 2X.  And first, 
we define a new style and turn 
off hyphenation for that style. 
We use the my bails style for 
load cases, and in version 21, 
we extend the base TextView 
style from the 
material.textview.  And finally,
on your team, you set Android 
TextView style to be your
new style.
And before I summarize, I will 
turn off hyphenation in your app
globally so your measurement can
be faster.  And it is not only 
going to be a text measurement, 
the whole screen measurement, 
depending on how much text you 
have, will be almost twice as 
much faster.  And then, please 
check the PrecomputedText API, 
and if you are using 
RecyclerView, please apply the 
PrecomputedText and RecyclerView
prefetch code that I showed to 
you so you get a more smooth 
scrolling experience for your 
users.
This is my talk, thank you very 
much for
listening.
Coming up next: Files for Miles:
Where to Store Them All? By Jeff
Sharkey SPEAKER: Hey, everyone, 
I'm Jeff Sharkey, I'm a software
engineer on the framework team, 
and we're going to be digging 
into files on Android and all of
the places that Android gives 
you to store those files.  So 
today we will start by looking 
at common locations that Android
offers.
  The first broad category is 
internal storage.  And this 
storage can be classified as 
safe and secure, because it is 
something that the Android OS 
protects and part of the 
application sandbox model we 
offer.  You have probably 
covered these directories 
before, like context file, it is
a great place
to store things.  
We will get into get cash dir, 
and the file space is not 
counted against the application.
Android reserves the right to go
in there and delete the files if
the user needs the disk space 
elsewhere.  It is a trade off 
and a two-way street.  Another 
directory, get- no back up files
dir, if you want to migrate 
between devices.  It is a great 
-- if the device goes through a 
back up and a restore phase, 
those files in the directory, 
they will thought  not be 
carried across the back up.  You
can use it for that.  
And finally, get code cash dir, 
for jitted and optimized code.  
It will delete under two 
conditions, when the application
is updated via Play Store, or 
whenever the OS itself has an 
upgrade, from the release to the
pre-release, it will clear the 
directory.  And another broad 
category of locations is 
external storage.  When we think
of that, it is more of a shared 
area and it is unprotected.
And the reason I just
mention that is data showed in 
that location, you may write it 
there, and other applications 
can request the storage 
permission on Android and they 
can write the data or modify it 
without you knowing about it.  
So it is something that we 
definitely discourage storing 
sensitive contents in that 
location.  If you need to store 
data there, consider finding a 
way to prevent and to verify the
integrity of that data if you 
need to trust it.  The 
directories here, similar to 
internal storage.  External 
media dirs are scanned by media 
store on the device.  It is a 
good place to store photos or 
videos included in the user's 
gallery application.  Get 
OBBDIRS, OBB stands for opaque 
binary blocks, they are large 
blocks for Google Play.  And 
data in those locations are 
counted toward the app's code 
size rather than data size.  So 
far, we have talked about 
internal and external storage.  
And these are all great places 
for you to store data that 
belongs to your app.  Be but you
might find yourself creating 
data that belongs to the user 
that the user may want to store 
in a different location.  And 
that's a great place to use the 
storage access framework.
And there are two intents that 
work great there, intent action 
open document and create 
document.  These have been 
around in the platform since the
KitKat release, you can think of
them as an open and a saved 
dialogue box for the user.  It 
really offers a great 
experience, because the user has
control over exactly where those
files are stored on the device.
It givesthem the freedom to 
choose any of those locations, 
and opens the door for cloud 
storage for  providers.  You 
don't have to integrate an SDK, 
you can launch the intent and 
the user can select where they 
want the file to be stored.  
There are great talks, I 
encourage you to search around 
online, there is great content 
that goes more in depth.  We 
talked about basic locations, 
let's do a deep dive on two 
scientific advanced locations 
today.  One is direct boot, and 
the second we will dig into is 
cached data.  The first is 
direct boot that you may not 
have encountered before.  And it
is worth starting out, we built 
the direct boot feature to solve
an important problem.  When 
before the end release, when we 
encrypted an Android device, if 
the user re-booted it, no apps 
can run until the user entered 
their credentials, a pin, a 
pattern, and password.  In the 
end releases, we created two 
storage areas, they are 
encrypted but with two different
keys.  And we call these areas 
the device protected area and 
the credential protecteded area.
The device protected area is 
available by virtue of the 
device proving it has not been 
tampered with.  When the device 
boots up, it verifies it has not
been tampered with and unlocked 
the protected storage.  When the
user enters the pin, pattern, or
password, it is available for 
applications to use.  If you 
have not encountered these APIs 
before, all of your data by 
default as an app developer is 
stored in a credential-protected
area.  If you find a place where
you would like to run before the
user has unlocked their device, 
that is where it might be useful
to store small bits of 
information out in that 
device-protected area so that 
your app can be useful while the
device is locked.  And then you 
might ask the question, how do 
you gain access to the storage 
area? 
Context.get-files.dir offers 
that's area.  And there's a 
method on context, create device
protected storage context, 
that's a little bit of a 
mouthful.  It returns another 
different context where the file
APIs, referring to internal 
storage, on that returned 
context point at the device 
protected storage.  So let's 
take a look at some code 
examples of how you might 
integrate with those APIs.  If 
you want to become device 
direct-boot aware, is to think 
about what data you want to keep
on credential protected storage 
or migrated.  So a lot of you 
are writing code like this, 
during the initial upgrade step,
how to migrate data back and 
forth.  The first thing you will
do in starting the application 
is to ask, is the user unlocked?
Has the pin, pattern, or 
password been offered? So you 
can check, is the current user 
unlocked? Assuming they are, 
that means you have access to 
the device and the credentialed 
storage. And here, you can see 
there are two move methods at 
helpers.  You can move the 
preferences and databases back 
and forth.  The reason we 
provide these helper methods is 
often times shared preferences 
and databases can actually be 
made up of multiple files on 
disk and some of that data may 
be cached in memory.  By calling
the helper methods, we ensure 
that all of the data is moved 
and any in-memory caches are not
invalidated along the way.  When
you think about data you want to
migrate, one thing we say is 
only move the data you need to 
provide the user experience 
while the device is locked.  So 
if you are building the alarm 
clock app, we will move the 
user's next alarm time into that
device-protected area to make 
sure that the alarm clock will 
go off if the user's device is 
currently locked.  Another 
strategy, if you have 
off-tokens, we recommend that 
people create a second type of 
authtoken for your full server. 
One ask a full access token, we 
recommend keeping that in the 
credential-protected area and 
creating a simple, limited in 
scope token auth token and only 
storing that in the device 
protected area.  And maybe when 
it talks to your server in the 
back end, it has three messages 
and cannot do operations beyond 
that.  It gets the user's 
experience when you need to, 
without accessing the richer 
information in the cloud.  So we
have talked, if it is unlocked, 
we can migrate data back and 
forth.  And the else clause 
here, there's a run time 
broadcast that is here, for user
unlocked, to run the code when 
the user enters that pin, 
pattern, or password.
And the middle code snippet 
here, the move methods, they 
work in both directions.  If you
move the data out into the 
device proteched  protected 
area, you can move it into the 
credentialed area.  And on the 
bottom of the screen, if you are
wondering if a particular file 
is going to be encrypted at REST
by the operating system, you can
quickly check for that as well. 
There's a storage manager, is 
encrypted API.  That can be 
helpful when you are rollering  
rolling your own encryption.  
And the second dive is cached 
data on the OS.  This is data 
that you can regenerate or 
redownload later on if it 
happens to be deleted.  This is 
a two-way street, the OS will 
not count the data that you use 
in that area against the app, 
but we reserve the right to 
delete that data if the user 
needs the disk space for 
something else they are doing.  
Something we improved in the O 
release, we rewrote the 
algorithms used internally.  A 
big question we got from you, 
from developers, is how much 
cached space is appropriate to 
use? Can I use 500MB, is that 
too much, is 50MB too much? Now 
we offer explicit guideguidance.
There's an API to figure out a 
cached quota for your 
application, that the device 
thinks is reasonable for the app
to use.  And it adjusts it over 
time.  If the user spends a lot 
of time in the application, we 
will increase the number so you 
can offer a better user 
experience to your users.
And another thing we did in the 
O release, we rewrote the 
implementation of how that data 
is cleared.  Before the O 
release, we list all of the 
cached files on the OS, sort 
them by modified time and delete
the oldest files.  There are 
ways to gameify the system, by 
setting the time out to the year
2038.  So we fixed that.  In O 
and future releases, the OS will
delete data fromp appessibility 
Partners s most over the the 
quota first.  If your app is in 
the cached quota that the OS 
recommended, you can be 
confident that the data is there
and will remain available as the
user starts filling up their 
disk.
And soso let's look at code 
snippets.  If you are 
integrating with a common class,
it is easy to connect the two 
things together.  We can ask for
the recommended cached quota 
bytes and plug it in to trim how
much size it is using.  If you 
have multiple types of caches, 
it is up to you to decide how 
you want to fractionally account
or distribute the cache against 
the internals of your app.  The 
second code snippet, if you are 
rolling your own caching, 
there's get cache size bytes, a 
fast way to ask the question, 
how much cached space the app is
using.  It is an optimized call 
that will return and
it should  it is faster than you
iterating over the disk usage to
figure out how much space you 
are using.
And another feature of the O 
release is the ability to have 
cached bihaviors.  We heard from
developers that it is useful, 
you may download multiple files,
it should be treated as a unit 
or a group.  You can download a 
movie and a sub titles file that
goes with it.  If you store both
in the cached directly, if one 
of them is deleted, the other 
file is not useful and valuable.
So the cached behavior allows 
you to tell us, as the OS, if we
need disk space, we should 
delete both of these at the same
time.  So we talked about the 
common storage locations, we 
will talk about how we can work 
together, both the OS helping 
you as developers.  And the 
biggest thing we offered in the 
O release is helping you get the
disk space that you need.  
Before the O release, in if you 
wanted to do a large download, 
and if you looked at the free 
disk space, you may only see 
500MB for free.  It may look 
like the download was 
impossible.  There is a new a.m.
tin the  API in the O release, 
it will delete cached files for 
the operation to succeed in the 
application.  If there isn't 
enough disk space, there are new
intents to launch to get the 
user involved, to pick items and
things they can do to help free 
up the disk space.  How to use 
the API? The very top of the 
slide, this is a way that you 
are writing code today.  You 
will do a simple check, a 
download size, Java.I/O.file/get
usable space.  That's an 
operation that looks like it is 
not possible to succeed.  If you
convert the code using the 
snippet on the slide, if you 
call storage manager.get 
allocated bytes.API, it will 
return not just the free space, 
but the space that the OS is 
willing to free up on your 
behalf.  So in this case it may 
look like, if we have enough 
space, we can actually open the 
file output stream and now there
is an API, allocate bytes.  And 
this will actually claim that 
disk space for your application,
deep underneath, it will use the
F allocate system call to make 
sure that the blocks
belong to the application and 
you have the space guaranteed to
you.
  And the else block here, if 
there is not enough space, we 
can ask the user for help to 
free up the disk space.
Sharing content, we covered this
before.  How can we work 
together there? Use content URIs
when you are sharing between 
applications, never use file U 
RRBS  RIs.  In sharing the 
application, you do not have the
permissions that you need to 
access the files on disk.  If 
you use content URIs, the OS can
make sure that they can open the
content.  If you find yourself 
in this position, the file 
provider is a great way to 
convert between the two with a 
single line of code, every place
to convert from file to content.
And over the years, because this
is an important thing to pay 
attention to, we built strict 
mode APIs to detect these places
in your app where you are 
accidentally sharing file URIs 
and in a more recent release, 
you can detect where you are 
sharing a content URI and you 
are forgetting the read or right
to go along with that intent.  
Those are two URIs that are 
helpful to look at.
And native code, that is another
area to think about.
And we strongly recommend that 
you look at opening files up in 
higher-level managed code, in 
Java or Kotlin, and passing the 
file descriptor down into the 
native code.  Opening a managed 
code gives the OS an opportunity
to notice, inspect, and correct 
things.  It can look for strict 
mode violations.  If you open up
the managed code, it may notice 
that the thread you are running 
on is causing a block or Jenk in
your app.  We are going to start
using this more in future 
Android release sreleases.  So 
open the file in
Java, past the integer, to the 
code.  Don't move it past the 
file itself.  And you can do 
this with parcel file 
descriptor, you can open up the 
file on disk, for read/write, 
and there's detach FD that 
returns the integer that is 
ready to cross against the J&I 
boundary.
And another method that is a tip
or trick that is useful.  If you
find yourself crossing against 
J&I to do a handful of system 
calls, look at 
Android.system.OS, there are 
several calls for you to use in 
Java today.  We added that a 
couple releases ago, so you may 
be able to FIMEGD  find that you
can do the sis calls in Java and
you can get rid of the J&I in 
the application. 
SPEAKER: And working with media,
we recommend that you use media 
store if you are looking to find
the files on your device.  You 
might be tempted to build your 
own index, that can be wasteful 
in terms of CPU and battery for 
the user. And we are working on 
improving media store and really
adding functionality there.  So 
stay tuned over the next couple 
releases.  And another note, 
open files, open the content of 
the media through content 
resolver.  There are columns 
across the operating system 
called _ data, they return a raw
system file path.  You may 
notice that a couple of the _ 
data columns have been 
deprecated in previous releases.
That is going to continue, you 
may notice those columns 
deprecated over time.  We want 
people to merge over to content 
URIs as a best practice.  Thank 
you for your time, in digging 
into the nitty-gritty areas of 
storage.  I will be in the Q&A 
storage after if you have 
equations -- questions for me.  
Thank you for your time.
[ Applause ]
Coming up next: Deep Dive into 
the ART Runtime by Nicolas 
Geoffray, David Sehr.
. 
SPEAKER: Everyone, the next 
session will begin in 10 
minutes.
SPEAKER: Hello, everyone.
I'm Nicholas from the Android 
platform team, thanks for being 
here.  I'm impressed that there 
are people showing up.  The 
Android fireside chat was kind 
of scaring me, everyone just 
avoided
the session.
It is great to have you here, 
this talk is about ART.  There's
a change in the schedule this 
morning, you may have attended 
Chet's talk.  Who went to Chet's
talk?
Okay.
How was it, was it good?
Yeah, were you excited?
18X, allocation
times?
So there's a part of this talk, 
that was supposed to be on 
garbage collection, and my 
colleague, David, was planning 
on giving. But we are not going 
to re-hash the same thing.
So I will put long pauses, 
awkward ones, I hope, during the
talk so I can fit it to 40 
minutes.  So bear with me.
So, given that we're celebrating
10 years of Android, I thought 
it would be a good idea to think
about what we've done the last 
10 years, and how the Android 
run time, the thing that we 
worked on for a couple of years 
now evolved.  So here we
are.
So some of you went to Chet's 
talk, so I guess you already 
know what an Android runtime is 
in the Android stack.  But in 
case you don't, it is the little
layer here between the Android 
framework, the Android operating
system, and the actual 
underlying Kernel.
The runtime runs both the 
Android framework, and all the 
apps, so everything between Java
that we
execute.
And being core in the platform, 
it is responsible for a ton of 
things, like the user experience
could be very bad if the run 
time was more efficient.  You 
saw it this morning with how the
GC was sort of, kind of, poor in
the
Dalvik days.
  I will show you that the 
runtime versions, the itRAISHZ 
itRAISHZ -- iterations over the 
past, what do we need to improve
for this year? Like I said, ART 
runtime is possible for a bunch 
of things.  And role performance
is one clear one, how fast we 
can execute Java code.  And 
clearly, it is also responsible 
for Jenk, the 16 milliseconds 
window, if the runtime is not 
able to execute the Java code of
the app, then we will miss the 
frame and produce a lot of Jenk.
The applications start-up, 
there's a lot of Java code that 
needs to be executed during the 
application start-up.  If the 
run time is slow, the start-up 
will be slow.
Boot times, a lot of code 
executes during boot.  Battery 
time, if you are slow, we will 
tank the battery.  And install 
time is something that we care 
about.  When we get an APK, the 
platform will optimize it and 
that could take a long time, 
depending on how we implement 
it.
And we don't want that long time
to happen, because we want you 
to use the app right away.
And the other tool, which is 
memory-relatememory-related, is 
disk space. 
It is how much space is the 
runtime taking for its own opt  
optimizations, and RAM.  Java 
being Java, is there an 
allocation that the runtime 
needs to handle.  If it doesn't 
do it well, it can take a lot of
RAM.
So this is essentially, there 
has been three incarnations of 
the Android runtime.
The first one was Dalvik, it was
the first implementation that 
shipped with Android.
And Dalvik's purpose, or 
Dalvik's main focus was how do 
we save RAM? And the reason 
being, back in the day, 10 years
ago, the RAM we had on the 
phones we're shipping was even 
less than 200MB.  And that was 
very little if you want to 
execute the whole Android stack.
So everything Dalvik was doing 
was about, okay, how it  do we 
save on RAM? So you cannot 
generate any code, JIT or AOT is
how we generate code.  It 
interprets the text code, what 
is sent to Android for execution
over the app.  Eventually, you 
get a Just in Time compiler so 
you can generate native code.  
But again, it was very limited 
to what it could do, because RAM
was the main focus.
And, as you see, this was 
tailored for apps not allocate 
objects.  If you have been to 
the talk this morning, things 
have changed.  But, back in the 
day, the recommendation was, 
like, please avoid allocations.
And this worked well for, I 
think, five years, until KitKat.
But there was, like, a point 
where, like, Dalvik could not 
keep up.  Phones were getting 
bigger -- phones were getting 
more performant, more RAM, like 
the -- that was 2013 or 2014, 
the 1GB, 2GB of RAM.
And apps were also getting 
bigger.
So initially apps were supposed 
to be like this small layer 
between the UI and the 
framework, but apps started 
doing a lot of more and more 
things.  So that 16 millisecond 
window I talked for rendering a 
frame, more was being executed 
there.  So we have to do 
something about it.  And the 
answer happened in Lollipop with
ART,
which introduced ahead-of-time 
compilation, and most of the 
things were ahead-of-time 
compiled, we were executing 
native code for the app.  And 
that is 20X faster than 
interpretation.
And we also introduced, like, a 
state-of-the-art GC, what you 
find in regular runtimes of 
being precise.  That means we 
are not going to be confused by 
an integer that looks like an 
object.
But also generations, so that 
the GC pauses we need to do in 
the UI thread will be very 
short.  So pauses will actually 
end up creating Jenk.
The third incarnation is like an
evolution of art, it happened in
two releases, like Android 
Nougat and Oreo.  In Nougat, we 
introduced profile-guided 
compilation.  I will explain a 
bit later what it is.  But it 
drastically helped on scaling 
ARTART's ahead-of-time 
technology to be more optimized 
for the platform.
The profile guide compilation, 
the way it works, it has a 
hybrid, just-in-time, 
out-of-time compiler.  We are 
trying to use the best of both 
worlds to optimize the platform.
And in O, after all of the 
optimizations in N, in O, we 
focused on the garbage collector
and implemented a band new one, 
which makes the pause
even shorter on the UI thread.  
We call it concurrent GC.  And 
now all the GCC happens on a 
different thread.  It is not 
affecting the execution of the 
app.
All right.  So before I dive 
into our details, I want to show
this, the state of the Android 
distribution today.
And, in case you are still 
optimizing for Dalvik and you 
still care about this annoying 
GCFORALLOC, there is still this 
10 percent, 10 percent is 
KitKat, Jellybean, about 10 
percent of devices are running 
KitKat.  So my recommendation is
it matters, 10 percent is 20 
million users, a big number.
So it still matters.  And give 
it a couple years, and hopefully
in two years, that will be gone 
and Dalvik can be part of
this museum.
All right.  So things ART 
matters for.  I have put eight 
boxes, they look nice, and we do
matter a lot for this.  If we do
get it wrong, things will go bad
on your device.
Role performance I talked about.
That's Dalvik's execution, Jenk,
application start-up, battery, 
disk space, RAM, boot times, I'm
repeating myself.  This is what 
is really important, what makes 
the user experience okay so you 
can enjoy the
apps.
So I'm going to go over the 
releases I talked about, the 
different incarnations of the 
Android runtime to show what it 
brings to the -- so what makes 
ART today.  Because ART has a 
lot -- like I said, it has a lot
of evolutions.  And we also took
good things from Dalvik.  I'm 
missing the major ones here, 
because the list is too long.  
And obviously, the major thing 
that Dalvik architecture brought
was RAM savings.
And, for that, Dalvik introduced
-- Dalvik on the Android 
platform introduced the Zygote, 
which is the parent process that
creates all of the other 
processes.
And because it is the parent 
process, you have the option of 
that parent process starting up 
or allocating a lot of memory 
for the apps to use, and that 
memory can be shared with the 
other apps.
And that is super important, 
that means that every app 
doesn't need to allocate this 
memory that it would need to 
actually execute in ART.
Today, I surround a couple dozen
MB that we save per app, and 
that the zygote allocates and 
shares with the other apps.
And then Lollipop was the major 
shift when we introduced 
ahead-of-time compilation.  
Ahead-of-time compilation 
happens with an SSA compiler, 
that's a compiler buzzword that 
is -- a state of the art 
compiler that does a lot of 
optimizations and makes your 
code up to 20X faster.
And we introduce the 
ahead-of-time compiler that 
helped a lot on reducing Jenk, 
now the code was compiled, not 
needing to be interpreted very 
fast, reducing application 
startup, the same argument, and 
saving battery.  The execution 
being 20X faster, you can 
imagine that it is not a point 
of saving 20X times on battery, 
but things are faster and we 
don't need to execute on the CPU
anymore.
We also save on boottimes.  The 
whole Android OS is ahead of 
time compiled and does not need 
to be interpreted at boot.  So 
things are faster at boot.
And we also introduced the 
international GC, which reduced 
the pauses for Alloc in Dalvik.
  And
in Oreo, we introduced 
profile-guided compilation, and 
that helps with all of the 
optimizations that we do.  It 
helps a lot of these metrics.  
It helps on Jenk, like less 
code, less compiled, and the 
things that we care about are 
optimized so the UI thread needs
to run less code.  And it helps 
on the application
application, we're able to know 
what matters at start-up so 
that, when we recompiled the 
app, we recompiled it with 
optimizations that optimize 
start-up.  You will have some 
battery.  Again, we're saving on
the amount of things we 
interpreted.
It will have some div space.  
Instead of compiling the entire 
app, which is what Lollipop was 
doing, now we are only compiling
the hard parts of the app.  That
means 10 to 20 percent of the 
dex code.  So 80 percent just 
doesn't get compiled.  That's a 
lot of savings.
It saves on RAM, having a 
concurrent GC means we can do a 
lot of defrag of the heaps of 
the app, so we save that on the 
fragmentation that we had on the
previous GC.
And profile guided compilation 
helps on the boot times.  
Remember the optimizing aches 
dialogue, we didn't need to AOT 
compile at boot all of the apps 
to make sure that the device was
reasonable and performant.  We 
were able to just, okay, we take
the OTA, we are going to jit all
the apps, when the user wants 
it, and then eventually we will 
do profile-guided compilation of
the apps when the user is not 
using their phone.
And then, finally, it helps on 
install times.  Instead of 
waiting for the compiler to 
compile the entire app when you 
install, now it doesn't compile 
at all.  We rely on JIT the 
first time the app was being 
used.
And, lastly, I wanted to mention
Pie.  The time we developed Pie 
was at the same time at Android 
Go, which was a great effort in 
the Android platform.  And we --
the work we did was mostly to 
save on disk space and RAM, 
because Android Go
is 5 or 12GB of the memory.  So 
most of our efforts were focused
on improving RAM and disk space.
So in that release, we 
introduced compact dex, which is
a compact version of the dex 
format.  That saves on RAM, 
because the less you need to put
into the memory of the dex code,
the more you are saving, 
obviously.
Also, when the APK has
uncompressed text stored, we 
will not uncompress it on disk.
So before, before we used to 
uncompress it to do 
optimizations on the dex file 
you cannot do on the APK because
it is signed.  So before pi, we 
uncompress it, do optimizations,
and rely on them on the first 
few iterations before we do 
profile-guided compilation.  Now
we get the option, give it to 
the developer, if the developer 
wants to save on disk space, 
then put the dex file 
uncompressed in DPK.  We are not
going to uncompress it on 
device, we have one version of 
the dex code and not a 
compressed version of the APK 
and an uncompressed one on disk.
All right.  That was a lot of 
optimizations.  I wanted to 
focus on one, which is raw 
execution performance.  What you
saw this morning was pretty cool
with 18X, this is even cooler.
So obviously, the faster ART 
runs, the more we are saving on 
battery on application on 
start-up and making the UI 
smooth.  It matters, all of the 
opt  optimizations we do.
And, over the releases, we have 
kept improving the performance 
by looking at actual 
applications.  In this case, it 
is the Google sheets.  And every
release we worked on, okay, how 
do we improve the Google sheets 
app.  And the Google sheets app,
or the sheets team, helped us to
build benchmarks how long it 
takes to do
sheets manipulation.
Here, hire is better, blow  blue
is Dalvik, it is a score of one,
relative to Dalvik in terms of 
performance.  Red is Lollipop, 
where we introduced ART.  And 
this is where we are today.  We 
went into a 4X improvement when 
we went to ART, to a 10X today 
or a 26X on benchmark.  We are 
happy with those numbers.  We 
didn't look just at sheets, we 
tried to look at what happened 
to other apps.
So a couple years ago, we also 
worked with the Chrome and 
YouTube team to see what they 
should optimize.  And there 
again, after the fact, even 
though we were not focused on 
optimizing those benchmarks, we 
saw that we had this 4 to 6X 
improvement with what we've 
done.
So there are two examples, the 
JavaScript benchmark suite that 
we imported for R purposes.  And
again, 2 to 4X, 3.5X for those 
benchmarks, up to 6X in Pie.
And
ExoPlayer, that's the video 
persister, driving the YouTube 
app on Android.  And again, 
around 2X for the introduction 
of ART and 4X today in Pie.
And while I have your attention 
on performance, I have a 
shamless call to do.  We are 
always super interested in 
improving code you think is 
important.
So if, on your side, you like us
to show off how we improved 
performance of your app, please 
come talk to us, there's the 
office hours from 1:00 to 6:00 
this afternoon, and we would be 
really interested in knowing 
what you think we should care 
about performance.
And we can show that off
here.
All right.
So the question, then, is as we 
get to this level of 
improvements, how?
I mentioned that ART has a 
modern compiler implementation, 
SSA.  And thanks to the modern 
SSA compilation, there's a bunch
of optimizations where we are 
able to do now.
If you look at the compiler, 
things can look 
familiar, with the debt 
coordination.  I will not go 
overall  all of them.  I will 
focus on an example that shows 
how the optimizations matter, 
especially on Kotlin that puts a
lot of abstractions to help on 
the produck tubty of the user, 
and makes it challenging for the
runtime to optimize.
All right.
So we will take the simple 
method.  Very simple, it is a 
function that takes one argument
and returns a length.
When we run
that to our dex serve, this is 
the code that you get.  Pretty 
straightforward, if you are not 
familiar with dex code, you 
create the string, and then 
Kotlin having Nonnullable types,
it will make sure that the 
string is not null when it is 
passed to the function, so it 
adds the helper method and 
checking that the perimeter is 
not null.
Then invoke field tool of the 
length method on the argument 
and return that.
Kotlin comes with the built-in 
library, so that's where you can
find implementations of those 
helper methods.  For that case, 
it is only a simple method that,
okay, is the argument null? Yes.
Then I will throw, calling in 
another helper, or R will return
and go back to the method.
So method calls are pretty 
expensive.  So the first thing 
that ART will do is try to 
in-line that very small method 
within the caller.
Here the compiler is in line 
with the place that it is 
called.  For simplicity's 
reason, it looks like the text 
code, it is the format of the 
compiler, but I will not show 
that to you.
Compile code is in line, which 
helps with the performance.
There is more that we can do, 
the compiler sees, oh, wait, 
that throw
parameter is null and it always 
-- throws.  There's a couple 
things you can do with that 
information.
The first is call layout, where 
you can put together the regular
flow of the method.  So things 
that rarely happen, you put it 
at the very end of the method, 
it does not affect the flow of 
the execution.
So we just switched the 
compilers in from, hey, are 
these zero, and we jump to the 
end of the
method.
So the jump is another picture
now.
The second optimization, we're 
going to move things that the 
regular flow does not care 
about.  In this case, let me 
just go back if I can.  In this 
case, the construction of the 
string that iss being passed to 
the helper was the first thing 
you executed in the method.  But
you only need that if you end up
calling the helper.  So-soo we 
move the construction of the 
string before the helper.  
Meaning, we don't need to 
execute examine.
So in the end, we started with a
method that was calling a 
string, a helper, doing it's 
thing, returning the length, to 
a method that is checking if it 
is null, if it is, jump to an 
expressive jump somewhere.  If 
it is not, we will continue the 
flow and return the length of 
the method, the length of the 
string, sorry.
All right.
So that was raw performance.  I 
have two other things to talk 
about, actually, just one, 
because I have to talk about 
application start-up and garbage
collection.  I will not re-do 
the garbage collection side, 
Chet did a great job this 
morning.
So, with application start-up, 
it has been a major focus since 
we introduced 
profile-guided compilation.  
That happened in nougat.  
Profile-guided compilation, when
the app is installed, we compile
it in a quick way.  We are not 
going to go to full IOT 
compilation, very little 
optimizations that do not affect
install time.  So we are 
optimizing install time.  The 
app is installed, then you run 
it.  The app is being executed, 
with interpretation, and then 
method gets hot, and then JIT 
kicks in and compiles the 
methods.
The JIT knows what those hot 
methods are, so we are going to 
jump to a profile file to sub 
methods. So when your device is 
idle, the user is not using it, 
it is charging, 100 percent 
charged, and then we have this, 
what we call, profile-guided 
demon that is just like, okay, 
then you walk over all the 
profiles and recompile the app.
And compile them, compile only 
the things that matter based on 
that profile.
And you have this loop where the
next time you run the app, we 
will use the optimized version 
of the compile code.  And then 
we will run it what is IOTed, 
maybe some methods are missed, 
you reinterpret them, re-update 
the profile, and the demon kicks
in, the profile was updated, let
me recompile the app.
So it is a virtuous loop of 
getting better and better over 
time.
And why is that helping on the 
application 
start-up?
The things that we compile in 
the app in the profile are 
optimized towards this.  We are 
going to compile the start-up 
methods.  So no need to 
interpret them, things that are 
executed at start-up will get 
compiled.
We're going to layout the dex 
and the compile code.  So things
that execute at start-up would 
be next to each other.
So now you don't need to jump 
over the entire dex file to get 
access to the method.  And that 
is very important.  Like I said,
apps got bigger.  So if you need
to bring up the entire dex file,
for start-up, that is a lot of 
time waiting on I/O.  So we try 
to reduce that by putting 
everything in start-up in the 
beginning and then the rest at 
the end.
Profile-guided compilation 
generates an application image. 
Other runtimes will call this a 
snap shot.  It is a 
representation of Java classes 
that we put in that image, it is
a file, and that avoids us to 
actually load the classes at run
time again.  So there's a 
pre-formatted number of classes 
with a class order, and when we 
start-up, you take the class 
order, all of the classes are 
populated, and we are done.  You
don't need to do code 
clustering anymore.
We are going to try to 
preinitialize classes, so Java 
says classes need to be 
initialized before executed.  
With profile guided compilation,
we need to initialize anything 
we can to prevent that from 
being executed before we start 
the app.  And finally, I said 
we're not going to compile code 
that doesn't get executed.  That
helps a lot.  Because then your 
old file is very small, your 
compile file is very small.  So 
there's not a lot you need to 
bring up in memory to actually
compute.
What do we gain from all those 
optimizationoptimizations?
Well, they can be -- we always 
gain doing those optimizations. 
Depending on the app, it can be 
10 percent or 30 percent.
And that is usually around how 
many Java code do you have when 
you start your app.  Typically, 
camera has a lot of native code,
that is on the low end of 10 
percent improvement.
And in this example, you see 
docs and maps, that are very 
Java-heavy, go from 30 percent 
of app start-up
improvement.
And this is numbers that we got 
from the maps team, who got that
from actual users, or actual 
data that comes from the field.
And when the maps team graphs 
what is going on, how come -- 
had they installed, things were,
like, a one second of apps 
start-up to, over time, things 
are faster.  How does that 
happen? And every time they 
update the app, it is the same 
trend.  It starts pretty high, 
and then it goes low.
And the answer is profile guided
compilation.  You are clearly 
seeing that, over time, things 
get better.
Today, in Pie, when we talked 
about I/O last year was the 
introduction of profiles in the 
cloud.  And that's how we're 
making the entire ecosystem send
us profiles, like actually 
execution of profiles for users,
so we can bring the profiles to 
new users of the app, so you 
don't get a -- the start of one 
second ends up 750 milliseconds,
they get the 750 right away, 
because they get the profile at 
the point of
the of 
install. .
  Garbage collection, I'm not 
going over it. 
Maybe I can just put back a 
number that we're all
very proud of, here we are, 
that's the last.  So this is 
resuming what Chet talked about 
this morning.  It is all the 
technology that we've used over 
time for building a GC.  So you 
see in KitKat,
we have this concurrent mark 
suite that was part of the GC 
that was concurrent.  And it 
stayed until Nougat, in Oreo, 
the concurrent collector.  The 
allocation in KitKat, it was the
bottleneck, it needs to lock to 
allocate something.  The 
introduction of a new GC in 
Lollipop meant that we can 
allocate within the thread and 
not need the lock.  
That improves the performance of
allocation.
The -- when you allocate the 
objects that are short lived, 
and that's the motoof Java Java,
feel free to allocate Java, it 
is moved by the GC in Java.  
Pre-KitKat, that is not the 
case, you have a very high cost 
by allocating temporary objects.
Lollipop is when we introduced a
new GC, and the -- you didn't 
pay that cost at all.  Like 
allocating a short-lived object,
we had generations, things were 
moved pretty quickly.  And 
there's an app series for Oreo, 
when we introduced concurrent 
collector, we removed the 
generations out of the 
collector.  We are fixing that 
today, it is in the AOSP, the 
improvement of the generations. 
And hopefully it will be there 
in the device
soon.
And fragmentation, fragmentation
is a big problem in Android, 
because if you are not able to 
allocate, the app will be 
killed.  So doing compassion of 
the memory, so that things are 
not fragmented is super 
important.  KitKat did a bit, 
but very little, and Lollipop in
marshmallow , we did it when the
app was running in the 
background.  We reclaim the 
memory.  But Oreo, it is 
important that we compact all 
the time so the memory is there 
and available all the time.
And the number I was looking 
for, the allocation speed, a 
very low number in Dalvik, to an
18X improvement in Oreo and
Pie.
And here are the reasons -- the 
reasons that we got improved, 
Lollipop added a custom 
allocator, and in Marshmallow, 
we had erfeweratomic operations 
that have a cost.  And all of 
that implementation of the 
allocation path was moved to an 
assembly code in nougat that 
made things faster, and then we 
implemented the pointer in 
allocation.  And the only thing 
we allocate is to increment a 
pointer.
All right.  With that, this is 
the recommendation that Chet 
has.
So I will give the same.
Creating garbage is
okay today.
You can use a tag to allocate 
the objects that you need.
GC is still overhead, so be 
mindful that if you allocate 
allocate a lot of objects, GC 
needs to run, but it is less of 
a problem since Dalvik.
And, with that, thank you.
[ Applause ].
SPEAKER: I will be in the office
hours at 1:00PM.  Feel free to 
come and chat there.
. 
SPEAKER: Everyone, please give a
warm welcome to dan Galpin. 
SPEAKER: We have lunch going on 
right now, we are on a different
schedule today.  So once again, 
lunch is where it was yesterday,
downstairs.  And we're going to 
start our sessions at 1:05.  We 
are trying to smash as much
--.
 #
Coming up next: Android Emulator
Deep Dive by Frank Yang, Yahan 
Zhou SPEAKER: Good afternoon, 
welcome to the Android deep 
dive.  I'm Yahan, this is 
fraFrank and we are software 
engineers from the Android team.
Back in 2015 in the previous 
Android dev summit, we announced
Android Emulator.  
So if you remember here is what 
it looks like in 2015.  And it 
is 2018 now, so things have 
progressed quite quickly, and we
have a new Emulator with a newer
look.
So if you look at the details, 
there's a few differences that 
you might have
noticed.
First, there's a frameless skin 
, and it gives you a better UI 
experience.
And secondly, you might have 
noticed that, in the expanded 
panel, there is a much longer 
feature list, with new features 
such as screen
recording and support, snap 
shot, and Google Play Store.  I 
will go over all these features 
in this talk one by one.
From 2015 to 2018, we have 
developed quite a lot of stuff 
in the Android Emulator, and we 
are focused on two areas.
One is the performance 
improvement of the Emulator, the
other is the new features.
And we are focusing on the 
improvng your day-to-day 
developing experience as far as 
CI server testing use
cases.
So, first things first, let's 
get into performance.  The first
thing we want to focus on about 
performance is HyperVisor 
support.  So just a background 
about Hypervisors, if you are 
running the Android Emulator at 
the lowest level, you have your 
computer hardware.
On top of that, you have your 
operating system, and the 
highest level, there
is the Android OS.  Between the 
OS and the operating system, 
there's the hardware which 
translates the Android CPU 
commands into your OS series of 
commands.  
That's the a part that we do the
CPU acceleration.  If you don't 
have a
HyperVisor, you can run the 
emulator, but everything is 
super slow and the emulator 
becomes super sluggish.
So, in 2015, here is the list of
the hardware that we support.  
On Linux, we have KVM, and on 
Windows and Mac, we support HAXM
with Intel CPU.  So 
unfortunately, back in the days,
if you are using a CPU on a 
Windows machine, we have no 
support for this combination.  
So we got a lot of request from 
A&D users, to add support for 
them.  So it is 2018 now, things
progress pretty fast, and 
obviously, we have new 
HyperVisors approach.  So this 
supports a new version of HAXM, 
7.3.2, which is the previous 
version and it gives you better 
performance.
And then we added 
HyperVisor.framework support for
Mac users which, in most 
situations, is faster than the 
HAXM on Mac.
And, at last, we support 
hypervisor framework,
known as HyperV on Windows.  So 
the combination of NDN and 
Windows is supportive,  .  So 
here is the HyperVisor 
performance.
And the next thing we want to go
into
will be the ADP performance.  So
the Oreo background, you move 
from the computer, on to the 
device when you are deploying, 
you are doing this ADB push 
command which copies and 
transfers the app from one end 
to the other.
So here, we are showing the 
chart for ADB push time on a 
physical device on two different
Google apps: The Google photo 
app, and Google center checker 
app.  And they are both on 
physical devices.  And you can 
see what the first thing was, 
that if you are using a USB 
cable and you are pushing the 
Google photo app, it takes 
approximately eight
seconds.
 If you are using a USB cable, 
it goes a lot faster, and it 
takes approximately four 
seconds.  So you can observe 
similar momentum when pushing a 
similar app, like the Google 
check
up.
USBKs are faster than USBK2s.  
So that's for the physical 
device.  How about the 
performance on the Emulator?
So here's a chart.  If you do 
the app push on Emulator, the 
Google photo apps take 
approximately 0.5 seconds, which
is a lot faster than USB2 or 
USB3.
And this other chart on an 
Android Oreo device. And it 
turns out that the OS version 
matters a lot as well when we 
are doing the app push.
So we've got a chart on the 
nearer Android Pie device.  If 
you are working on a physical 
device, the ADP push is a lot 
faster on either USB2 or 3.
And Emulator still gives you a 
very fast and a very constant 
performance.
So, in short, if you are going 
to the emulator, you are not 
going have any worries about USB
cables or
Android versions.  You can 
always get the best and 
consistent performance.
The next topic, we are going to 
Quick Boot and snap Shots.  We 
are showing demoes.  And with 
that, we will have 
Frank. 
SPEAKER: Thank you, Yahan.  A 
top concern of Android Studio 
users is time to deploy the app,
whether it be on a physical 
device or emulator.  And people 
are used to the workflow where 
they are inside studio, want to 
deploy the app, and they want to
wait for the Emulator to boot up
again.  We tried to address this
in 2017 and in 2018.
The first way we tried to 
address this is through snap 
shots.
We call this quick boot.
With quick boot, the device 
state is completely saved when 
you take a snap shot.  And when 
you take a snap shot when you 
exit the emulator and you resume
from that snap shot when you 
start the emulator again.
So, in a sense, it is kind of 
like sleeping and resuming a 
real device.  And we've also 
made improvements in recent 
emulator Canary versions to make
it even closer to that.  So the 
demo I'm going to show you here 
is I have two ABDs set up in the
ABD manager.  One of them is 
called API 28, and we're going 
to be using quick boot with 
that.  And a second one is 
called API28 cold boot, and 
we're going to cold boot that.  
So for our benefit, people 
haven't seen this already.  I'm 
going to cold boot one of these 
ABDs.  You will see that it 
starts out as a black screen, 
takes a while to show the Google
logo but, with quick boot, the 
Emulator pops up right here.  
You will notice that the log cat
started up, and we can interact 
with it immediately, just as the
Cold Boot emulator has just 
started up.
So we are going to get rid of 
the Cold Boot emulator for now 
and move to the next part of the
demo.  In addition to Quick Boot
in July of 2018, we extended the
snap shots feature to be more 
general.  So with quick boot, 
you can only have one snap shot 
being saved and loaded at any 
one tile.  Time.  That is 
usefulx , but maybe you want to 
test different device 
configurations and state and so 
forth.  Before now, we didn't 
have a good way to do that.  In 
the latest Emulator stable 
version, 27.3.10, released July 
2018, we are giving you a 
generic snap shots UI.  In this 
generic snap shots UI, you can 
save and load snap shots 
arbitrarily.  So for example, I 
can immediately 
resume one of these snap shots 
to stop watch running, and a 
stop watch is running.  I can 
edit any one of these snap shos 
and add notes.
And the description is saved and
also displayed over here.
I can also take new snap shots 
with the take snap shot button.
So these are some examples of 
what we have been doing with 
snap shots in trying to make 
your life easier when you are 
deloying apps and testing them. 
I would also like, at this 
point, go over some improvements
we made to the quick boot 
mechanism itself in recent 
emulator canaries.  The version 
of the Emulator running right 
now is 28.0.15.  And the recent 
Emulator car nary, we tried to 
listen to your concern about 
extremely long saving time for 
the Emulator when you are 
closing it.  Now if you want to 
save the Emulator, actually, 
wait, we will set up the stop 
watch again.  It is saved, trust
me, it is.
Yes, it was saved.
So we have employed new 
technologies, like file backing 
of the guest RAM in order to 
speed this process up.
And in addition, we have also 
added some console commands that
are useful for CI users when 
using snap shots.
One very common workflow is to 
reset the device to a particular
previous state, and then run 
your test in  and reset the 
device again.  To that end, we 
added this command called, in 
the console, it is called ABD 
snap shot remap 0.
So assuming that you are using a
default setting, it is like you 
are auto saving the current 
device state all the time.  So 
right now, we are on the second 
-- we are up 40 seconds. 
So the 1st time you run this, it
will save it around then.  The 
next time you run it, it will 
quickly rewind there.  So this 
happens quite quickly because 
all the rest of the Emulator UI 
and all of that is set up, it is
just a device state that is 
changing.
So you can just keep doing this,
like rewinding very quickly, you
can be unany state of the device
and just keep going back to that
one state very quickly with this
command.
And another feature that we have
been working on in recent 
Emulator canaries is launching 
multiple instances of the same 
ABD.
So let me just set this up 
again.
Okay, zero seck  seconds.
So before, when you are using 
the Emulator.  If you want to 
test multiple different device 
dates and launch them all at the
same time in parallel, your 
options are pretty limited.  You
have to create a bunch of ABDs 
that reflect the different 
device settings and launch them 
all in parallel, or you could 
start two instances of the same 
ABD -- couldn't -- because you 
got the same message that the 
ABD is in use.  We have a 
command for that now, read only.
Where  I will show you the 
command here.
Okay.
Actually, it is missing, but I 
will type it again.
So it is called dash read dash 
only.  So when you run this 
command, the emulator will 
start, but the start all 
changes.  I am starting it in 
the background to make it easy 
to start many of them.
So I have just started a second 
one, it is actually overlapped 
on the first.  A third one u , a
fourth one.  So you have these 
ABDs that are running 
independently, but off the same 
ABD name, because I used API 28 
for each of them.  And I'm 
cleaning up the FPS and memory 
stats.  Why are some of these 
only 200 or 300MB of resident 
memory? That goes back to using 
a file mapping for the guest 
RAM.  In this case, the parts of
the guest memory that they share
in common that they have not 
modified yet will be shared and 
will not occupy proportional 
size or resident memory or 
what-have-you.
So this is called copy on write.
And we hope that this makes your
experience developing apps and 
testing them more streamlined.
Thank you.
Now back to Yahan, back to the 
slides. 
YAHAN ZHOU: I think it is 
obvious to see the comparison if
you are using the quick boot 
feature versus if you are doing 
the cold boot.
So, in the cold boot, if it 
takes you more than 20 seconds 
to do the save on your machine 
as you are demoing today, and on
Quick Boot, it is usually about 
one second.
So back to Frank for the next 
demo.  
SPEAKER: Uh-huh, back to the 
demo.
So the next part starts out with
we also improved the GPU 
performance of the emulator in 
recent versions and in recent 
years.  We worked a lot to bring
open GL ES2 support very 
conformant and to support open 
EL GS3.
So this end, we created some 
benchmark demo apps that 
showcase what the emulator is 
capable of.
So I am starting this test over 
here.
And so you can run the realtime 
3D applications like this that 
are competitive with real device
performance.
But, before we go any further, I
would like to go back to Yahan 
in the slides to go over another
interesting feature.  
SPEAKER: All right.  So, coming 
back to the Snapshot support.  
If you remember in 2010, the 
emulator, before 2.0, we had the
quick boot feature and snap shot
tutorial.  At the time, the 
emulator supports snap shots for
RAM, Disk, and CPU.  The missing
part here was GPU-based snap 
shots.
So this eventually led us to 
choose between hardware GPU 
emulation and the quick boot 
feature.
If you choose the hardware GPU 
emulation, you getty pretty nice
runtime performance.  But your 
boot time is a slow boot.  If 
you choose the quick boot 
feature, your boot time will be 
fast, but you will have no run 
time hardware GPU support, which
means the runtime performance 
will suffer.
So this is in 2010, and people 
have had to choose between the 
two.  And it is 2018 now, and we
have better versions of 
everything, including the 
hardware GPU snap shot support. 
This allows our users to choose 
both the quick boot feature as 
well as hardware GPU emulation 
simultaneously.
So if you look at what was 
happening behind the scenes for 
this hardware GPU snap shot 
support, here is the emulator 
GPU stack.  At the highest 
level, you have the Android OS, 
which issues GPU commands.
At the lowest level, you have 
your hardware graphics card, 
which sets the commands.  And we
have the emulator, called the 
translator.  And the translator 
will translate the GPU commands 
from Android into your hardware 
GPU.
And when we are doing the GPU 
snapshot support, we put all of 
the logic into the translator 
layer so that if you serialize 
this own state and dump them 
into your snap shot saved file.
So one big question here was the
hardware states, which is
eventually over to the emulator 
and different renderrors render 
differently.
So to use the hardware state, we
standard the APIs to fetch the 
hardware space from the hardware
to the translator layer, and 
then we serialize all of them 
and wrap it into the snap shot 
file.  In this case  in this 
case, we can save and recover 
the whole emulated hardware GPU 
state.
So -- 
SPEAKER: All right, thank you
Yahan.  In this demo, I will 
demonstrate that the GPU state 
is snap shotted in its entirely.
We did not do something special,
it can resume 3D apps like this 
almost immediately with the 
snapshot load feature.
All right.
And you can, for those of you 
who don't know already, you can 
rotate the emulator in rotation,
using the rotation buttons.  
Back to the slides I would like 
show a brief presentation.
All right.  So I know that many 
of you know this already, but 
for the benefit of people thin 
audience who don't know yet, I 
would like to recap on some of 
the basic ways to use the 
command line to launch the 
emulator, which can include 
useful options if you are 
running a CI server, or power 
user.
  So the emulator and the 
Android SDK folder, Android 
SDK/emulator.  And in the 
directory, you have to give the 
ABD command line option, may be 
different than what is displayed
in the ABD manager.  If you want
to know the exact sequence of 
characters to input there, then 
you run 
emulator-listABDs.
Second, how it  do you control 
quick boot.  Sometimes you don't
want to use snap shots or save 
your changes
on exit.
So the first snap shot is to 
attempt to load the current 
state on the device, cold boot.
  The second, quick boot, it 
dischards changes to the snap 
shot for CI use cases and if you
want the consistent start up 
experience and want to wipe it 
clean when they are done with 
it.  In this, you enter no snap 
shot save on the command line.  
And all of the changes to the 
snap shot will be discarded when
you exit the emulator.  Finally,
if snap shots are not your 
thing, you can add no snap shot 
to your command line.  That will
bring the Emulator back to the 
way it was before.
And to recap, in the demo I gave
earlier, in launching multiple 
instances of the ABD, that's the
read only option.
That option will control whether
you are allowed to run multiple 
instances of the same ABD or 
not.  When you are running in 
that mode, all of the changes to
both the guest virtual disk and 
to the snap shot, whatever you 
do, it is discarded when you 
exit.
So invoke this, you would run 
emulator-ABD, the AB it  D name,
followed by read-only.  I put in
an ampersand, you don't have to,
but convenient for demo purposes
here.
All right.  Now, I want to talk 
about some of the command line 
options super some of the 
features that we have introduced
recently to the emulator.
In particular, screen record.  
Screen record is a feature where
you can record the contents of 
the emulator to a video file, 
and then play them back using 
your favorite app.
This is great for CI users who 
want to be able to easily record
what is going on when running 
tests on their app, and to 
easily send reproductions of 
bugs to stakeholders.
So the first set of commands 
concerns how to start and stop 
the screen recording.
 ADBADBemu screenrecord start, 
followed by the path to the 
file, will start the recording. 
And it is important that you 
provide the absolute path.  
Otherwise, you might not get it 
saved.  It is a little 
confusing.  Sorry, working on 
it.
The second command, ABDemu 
screen record stop, it stops 
recording and there's a menu to 
save the file.  So after you to 
it do do that, you can access 
the file. And the third one 
concerns if you don't want the 
whole video recording part, you 
can just instead take a 
screenshot.  In this use case, 
the console command is screen 
record screenshot, followed by a
path to the directory where you 
want the screenshots to be 
saved.
Now, let's go to the demo where 
I will demonstrate the screen 
recording commands.
So we are in Emulator right now,
and let's try to find a 
different app to record, we will
start up the camera.  So while
that is going on, I'm going to 
run this, so ADB emu, screen 
record, start, users, user name,
documents, they have summit, 
demo.webm.
So while it is recording, we 
will just move around in here a 
little bit to make something 
happen.
And we will stop the
recording.
And that is kind of it  -- I 
accidentally typed ADB there.  
So we will do another
recording.
All right.  So the video is 
saved, and you can open it in 
your favorite web
browser.
Okay.
Let's see, there we go.
So it is a Web M file, it will 
work in most major browsers.
Second, there's always an UI for
recording these screens.
It is in the extended controls 
under the screen record tab.  So
in the screen record tab, it 
starts with a single button, 
start recording.  So once we are
starting recording, we will move
around a little bit, go to 
recent, swipe away, and stop.
And now once you are done 
recording, you don't have to 
save it to a file.  You can 
preview it directly in the 
extended controls UI.
So it is just a video of what we
just did.
Go to recent, swipe away the 
apps, go back to the home 
screen.  But you can still save 
it wherever you want.
So let's just save this as demo 
two.  And that, similarly, works
in the browser.
Okay, so that's one of the 
features that we've added in 
2017 and 2018 is screen 
recording to make your CI use 
cases better.
Another feature that we've been 
doing a lot of work on and it is
pretty exciting is AR core 
support.  Now, AR core support 
consists of a few mage 
components: The first one, as 
you may have seen already, is 
the virtual scene camera.  With 
the virtual scene camera, we 
render a virtual environment, 
which is suitable to image 
recognition and model placement 
in AR apps.
So I will demonstrate a few of 
them here.  In the back ground, 
in Android Studio, I have opened
the augmented image, AR score 
sample.  This is available in 
scene form Android S it  DK from
GitHub.  And what AR core 
provides, and it is not just for
the emulator, but integrations 
in studio as well.  So, for 
example, there's this scene form
plugin for Android Studio where 
we can view models that are 
meant for recent error core 
apps.  In addition, we can 
change all the material 
properties we want in studio.  
So this is the lower left frame,
and I might just say, lower the 
red tint
a bit.
I guess that did not change the 
color very much, so I will 
change blue to zero as well.
So once you save these files, 
the model should get reloaded.  
And you will see that it is 
mostly green now.  And the 
workflow, we can edit the model 
and deploy it to the device 
while we wait,
I guess.
All right.  So, in this app, we 
are looking for images to 
recognize.  Another feature that
we have added to the emulator 
for AR core support is the 
ability to load augmented 
images, this is in the extended 
controls panel under the camera 
tab.  We can choose different 
posters to appear inside the 
virtual scene, and I have set 
this JPEG of the earth to appear
on the table and the wall.  So 
once you travel inside the scene
with this, and turn, face the 
wall, we are starting to detect 
the image and it shows up there.
So this show physical s if you 
are a developer of AR apps, you 
have an end to end workflow and 
you can do it with scene camera.
So that's pretty cool.  Finally,
another feature I would like to 
mention that I will call out, 
unless you haven't seen it 
before, is improved support for 
updating GMS core in Google Play
services.  It is important to 
app developers because you have 
to test how the app will run 
with different versions of core.
If you see the image in the 
controls panel, there's a tab, 
Google Play, and inside it, 
there's a button that shows the 
current GMS core version and a 
button to up late it.  Once you 
press update, you will be take n
tothe Play Store and be given 
the option to update it.  I hap 
happen to have it updated 
already, it is the latest 
version.  If a new version does 
come out, it would say update.
I would also like to talk about 
the integration of the Play 
Store and the Emulator in 
general.
So this can be useful for app 
developers who would like to 
test their app in more 
end-to-end scenarios.  If your 
app requires other apps to be 
installed in order for it to 
work, you can download them from
the Play Store directly.  Also, 
you can look at your app in the 
marketplace and see how users 
are reacting to it right there 
and from different device sizes 
in different form factors.  And 
so that concludes the feature 
demoes.  So back to Yahan in the
slides. 
SPEAKER: All right.
So here is the summary of the 
features we talked about today. 
You might have noticed that 
there are two channels of the 
emulator out there.  The first 
is the stable channel that you 
normally get.  The latest update
is from July 2018.  The second 
channel is the canary channel, 
which is less stable, but has 
all of the cutting-edge features
we have in the emulator, and it 
is released weekly.  Feel free 
to try it out.  In the stable 
channel, we have this feature, 
this is already released, 
including HyperVisor support, 
quick boot feature, generic snap
shot SFEECHer, the Play Store, 
the screen recording, and the AR
core and the virtual sensor 
support.  If you are trying out 
the canary channel emulator, you
have a few more steps, including
the faster quick boot feature, 
as we have shown in today's 
demo, as well as the 
multiinstance support.
And for those of you who are 
curious on how to switch the 
channel, you can create the 
instruction.  In the Android 
Studio, you can turn on your 
preference page.  In the update,
you have your channel switch 
right there.
And after that, you go to the 
emulator download page and use 
the newer version available for 
download.
So I believe that concludes our 
talk.
So we will be QAing in the 
laundry, so thanks for attend
attending.  Thank you.
[ Applause ] 1QU #
St.
Coming up next: Vitals: Past, 
Present & Future by Fergus 
Hurley, Riccardo Govoni.
C  SPEAKER: Hey, everyone.  
Thank you very much for joining
today.
I'm Fergus Hurley and I'm 
excited to talk to you about the
past, present, and future of 
Android V  vitlsals, Google's 
initiative to improve the 
stability and performance of 
devices.  We are doing a lot to 
help this, in the play console, 
we have section, Android Vitals.
That's where we report to you 
different performance metrics we
are going to cover in this
talk.
How we empower this is 
interesting, hundreds of 
thousands of users share their 
diagnostics data, and we share 
that data in a privacy-compliant
way within the product.  I will 
start by going through lessons 
we learned over the last 18 
months as we launched vitals 
from top developers out there 
that are big and small that have
actually used Android Vitals and
gotten a lot of success from 
using them.
So hands up here if you want 
less one-star ratings? If you 
don't have your hand up, you 
might be in the wrong 
conference, unfortunately.
But most people want to avoid 
them.  What we found when we 
looked at the review corpus on 
Google play, which is pretty 
large, is that 40 percent of all
one-star reviews were extracted 
and talking about stability in 
bugs.  Over the past year, the 
percentage of users talking 
about stability has gone down 18
percent.  We are going to talk 
about the developers that 
contributed to this
reduction.
The emerge dragons team has used
Android Vitals to pin point 
where there is an issue with 
their application and this 
resulted in them reducng their 
one-star reviews by 50 percent.
The very popular Reddit app used
Android Vitals to be able to 
reduce their crash rate by 75 
percent.  They did this because 
they were able to see the 
category benchmarks in the play 
console Android Vitals section. 
And they were able to make the 
case with their leadership that 
they had room for improvement, 
and then when they actually 
improved their metric, they were
able to show leadership, hey, we
have actually done what we set 
up to do and we are one of the 
most stable apps out there.
The logo team signed up for the 
anomaly emails, and they got 
notified when there was an issue
in their application.  They had 
a huge spike caused by an SDK 
they were using in their 
application.  And even though 
you might be building an amazing
product yourself and think it is
stable, I'm sure you have used 
third party libraries and some 
of them can introduce issues in 
your application.  You want to 
become aware of that as soon as 
possible.  And anomaly alerts 
distable it, and then they 
shifted it to a new fix that 
reduced their ANR rate down to 
0.25 
percent.
The AVA English team, pretty 
small and they don't have 
infinite resources to spend on 
performance.  We don't think you
should spend all of your time 
focusing on performance, either.
But you have to keep an eye on 
performance all the time and it 
might make sense for you to just
focus on improving the 
performance of the application 
and hold off on the new 
features, maybe as part of
the '19Q1 planning.  This team 
did that and reduced their ANR 
rate by 97 percent as a result.
This, and a bunch of other 
things, resulted in them being 
able to increase their star 
rating to 4.6 stars.  You cannot
get a 4.6 star reading if you 
just focus on vitals, but you 
will never get a terrible one if
you focus on vitals.
  We looked at individual app 
users using vitals.  We also 
looked at, of the top thousand 
apps on play, we looked at their
engagement in the vitals 
section.  And the developers who
engaged most in the vitals 
section, the top 10 percent, 
they reduced their crash rate by
50 percent over that time 
period.  And this isn't just 
good for those developers and 
those apps, they are good for 
Android end users that resulted 
in $3 billion more stable 
installs on the
platform.
So we talked about stability, 
and another major area that we 
just talked about when they 
leave one star reviews is 
resource usage issues.  And this
is
battery, network, memory, etc. 
We have a lot
interrelated.  One is excessive 
network usage in the background
background.  It is consuming the
data plan and wasting the 
battery.  We have two metrics to
help in this area, wake ups and 
wake locks.  And the team 
focused on their wake ups rate, 
and they said they wouldn't have
even noticed or fixed their 
excessive wake ups issues 
without using Android Vitals, we
have a lot of insights in your 
battery usage that few other 
tools can help you with, and 
they reduced their wake up rate 
by 70 percent.
Now we're going to jump over to 
how you can actually increase 
your 5-star reviews.  And what 
we find is over the past year, 
we've had more users mentioning 
speed, design, and usability, 
which is what 70 percent of the 
five-star reviews talk about 
when we are able to find topics
in 
them.
The mercado Libre team, number 
one for eCommerce in Latin 
America, they built a new 
feature in the application, 
requiring them to ask for a 
certain permission, resulting in
a bad user experience where 
users had a high denial rate of 
this permission.  They would see
it in Android Vitals and 
redesigned that feature to give 
users an explanation of why they
needed it, that permission, and 
only ask when the users actually
needed that for that specific 
feature and not early on in the 
process.
And Mercado Libre team is pretty
big. And what they found useful 
is the ability to keep track on 
many of the metrics and to see 
that -- the start-up time, 
there's was a regression in 
reduces and that caused the 
start-up time of the application
long.  They found out the team 
that caused the issue and worked
to debug that 
issue.
This team used the monitoric ing
tool to get deep am  insights.
And it does not cover in the 
application where the issues 
might occur, so that's why we 
encourage you to use Firebase 
performance monitoring as well 
to understand where the specific
issues are occurring.  They 
figured out that the log in and 
sign up flow is taking too long 
and they decided to re-write 
that part of the application.
So what's in vitals today?
So vitals consists of five 
performance areas that you will 
agree are things you don't like 
have happen when using apps, you
don't want it to crash, drain 
your battery, slow in rendering,
requesting permissions you don't
need, and slow to start.
And so we have 15 metrics 
covering these
these five performance areas 
today.  And we showed you an OS 
version, your APK breakdown, 
device breakdown and, where 
relevant, we provide other 
information as well when it is 
possible.
And a new feature that we 
launched at I/O this year is 
category benchmarks where you 
can see, compared to other apps 
in your category, where you 
stack up compared to your peers 
across every metric.  We give 
you the 25th, 75th, and 50th 
percentile break down for the 
category benchmarks and that 
helps you to understand, this 
metric I'm doing okay on, and 
this one I'm actually falling 
way behind and need to invest 
there.
I mentioned the anomaly alert 
emails earlier.
These are available in the 
console today, you have to opt 
in to receive them.  So I 
encourage you, if you are not 
opted in already, to sign up 
today and I encourage the rest 
of your team to sign up as well.
And this is where we notify you 
when there's a significant 
change in the ANR crash rate of 
your application, where you have
a spike in the clusters, or 
across the core vitals that 
we're going to talk about right 
now.
There are 15 Android vital 
metrics, it is a lot of metrics.
We think all of them are 
important to deliver the best 
possible user experience, but 
some of them are more important 
than others.  As talked about 
earlier, the leading 
contribution to one star 
reviews are civility and baltry 
issues. 
THROOPS are four vitals across 
those, crash and ANR
rate, and stuck partial wake 
locks and excessive wakeups in 
the battery area.  For each core
or vital, we give you a bad 
behavior threshold.  If you are 
above that threshold, you are 
sort of failing the class.  That
bad behavior threshold is 
looking at how it maps, the 
bottom 25 percent of apps are 
above this rate, and that's 
where the bad
behavior is.
We launched the pre-launch 
report a couple years ago, this 
is when you upload your APK, we 
would generate a report within 
an hour of how your application 
performs across those different 
10 Android devices that you have
in the test lab when a robot is 
navigating your application for 
10 minutes.  It gives you a 
report showing a screenshot 
clustering and a bunch of 
insights into accessibility 
issues and privacy issues and 
also flags crashes.
One we recently did is we 
enabled you, when you look at a 
crash cluster in the vitels 
section is to see if we have a 
pre-launch crash report that we 
detected that matches the 
pre-launch crash report.  And 
there are not the same privacy 
restrictions in place as there 
are with the data from the 
field, of course.
And so here, we can be able to 
show you the video of the robot 
interacting with your 
application so you can quickly 
reproduce that issue yourself 
and we provide a lot more log 
and detailed information.
We also did the reverse.  So we 
have the ability to be able to 
see when you are looking at a 
pre-launch report, is this a 
crash that is already happening 
in the field, or is this one 
that was introduced with my 
latest APK and has not reached 
users yet? 
SPEAKER: We have over 100 
developers using Android Vitals 
today.  Hands up here, who is 
using Android Vitals? Cool, 
great.  Thank you guys for using
it.
And one of the things that I 
found most interesting about 
this project is that we have 
expanded the number of users who
are using performance metrics, 
or what we call engineering 
metrics.
So I was previously working on 
apps with another team at 
Google, and we did that external
to Google before. And one of the
thing tat I struggled with is 
understanding how these metrics 
are doing.  We had to have the 
latest version of the code, and 
to be able to run the profilers 
and we made it so it is really 
easy for all the people in your 
company to get access to these 
performance metrics and be able 
to see how you are doing 
relative to the other people and
make a case for investing here, 
versus the other features.  I 
have shown some case studies 
from different titles here, as 
you would expect, you have an 
Android engineer, a head of 
mobile engineering and a product
manager using it.  And now we 
have lead performance managers 
using metrics, you have a CTO 
and COOs using engineering 
metrics.  I encourage you and 
your teams to use these with 
your senior leadership and with 
the wider team.  And if you 
don't, you are probably going to
get left behind.  Because 
quality across the whole Android
ecosystem is improving.
So you would say, that is great,
it is impacting the other 
developers, not me, my app is 
awesome, even if it has a high 
crash rate.  I don't believe 
you.  We looked at the data of 
users with a high crash rate 
versus a lower or medium crash 
rate.  If they have a higher 
crash rate for the same app 
leave 52 percent more reviews 
than a low or medium one.
So I encourage you to use it if 
you are not.
So what is the future of Android
Vitals?
Well, we have built a lot of 
features, as you can tell so 
far.
And so really the feature of 
Android Vitals is you guys are 
using the product.
And so, what we're going to do 
is right it now, we're going to 
try something different.  I want
everyone to stand up, and talk 
to the people beside you, so try
to find -- there you go.  Yep.
And find someone beside you who 
hopefully you have not worked 
with and just introduce yourself
and we're going to take two 
minutes for one person to talk 
about their vitals and how they 
are approaching it, and who in 
their company uses it, what are 
the best practices involved, and
then switch the role for two 
minutes.  I will give you two 
minutes now, and then we will 
switch over.
SPEAKER: If you haven't had the 
other person talk, switch over 
and we will have one
more minute.
SPEAKER: Thank you very 
much, everyone.
I hope you learned something 
from the people you talked to, 
and made a new friend as well.
So I would encourage you to talk
to other people about how they 
are approaching their vitals.  
There's lot of documentation 
online to help you as well, and 
we will be at the booth outside 
afterwards on the right hand 
side when you walk out of this 
room if you have any questions 
as well.  One of the things that
you probably realize talking to 
the person is that one of the 
biggest problems with vitals 
today is actionability, we heard
you and your feedback, and we 
appreciate that.  And I want to 
show you that is something we 
are working hard to improve and 
we have more to come on that in 
the future.  We do believe that 
the data is valid in terms of 
telling you the direction to go 
as in, like, you are doing good 
or bad versus other people.  And
we think we can do better in 
terms of helping you be more 
efficient at fixing the problem.
But that is a really hard 
problem, and we are working on 
changes to the whole OS it elf
itself. 
Those things take a while to 
propagate across the Android 
user base and to be able to have
enough data to share back with 
you within the console is 
something we are working on.  I 
want to thank you all very much 
for joining us today, and best 
of luck with improving your 
vitals.  I hope I get to share 
your vitals' success stories in 
our talk next year.  Thank you.
[ Applause ]
.
Coming up next: Re-stitching 
Plaid with Kotlin by Florina 
Muntenescu.
SPEAKER: Hello, 
everyone.
I'm Florina Muntenescu, a 
developer advocate at Google.  
For the past few months, me are 
working on bringing Plaid in 
fashion, it was brought by a 
colleague of a way of showcasing
material design.  We are seeing 
the state of the
app in 2016 in Plaid glory days.
We see animations and 
VectorDrawables and a lot of 
these made the app shine and 
improved the user experience.
So Plaid takes data from three 
different sources.
And, well, these sources from 
2016 until now, well, some of 
them were deprecated.
So that meant that, out of three
sources, we were left with one 
and a little bit.
So, from all of these nice, 
fancy UI features that we had, 
we were left with almost nothing
and with a pretty boring 
application.
So we decided we didn't want to 
leave it like this, and we said 
that we wanted to fix this 
broken functionality.
But apart from this, we also 
knew that we wanted to go 
towards something that a modular
and expenseable from the 
architecture point of view.  But
the thing is that Plaid was 
developed as an UI sample, not 
as an architecture sample.
So you won't be surprised to see
all the dependencies in the code
and, well, code that was 
actually a bit behind, because 
Nick started building this in 
2014.  At that time, we didn't 
have the guide to that 
architecture that we released 
last year.  And also, although 
we have Kotlin, we doesn't 
really use it.  So we knew that 
we wanted to rebuild Plaid and 
in the right way.
So to have it in a good state 
for any future changes that we 
wanted it build.
So in this talk, I wanted to 
tell you what we've learned and 
how we managed
to leverage calls in our 
app.  We released this guide to 
the app architecture, and it is 
still quite vague.  So we have 
these main classes, some idea 
for how we should architect our 
app, but we wanted to create 
clear guidelines.  So we defined
main types of classes in our app
and we also defined a set of 
rules for each one of these 
classes.  So we defined a remote
data source, whose role is 
actually just to store the 
request data, fetch the data 
from the API service.  And 
that's it.
We could request information and
return the response received.
  Next, we would have a local 
data source, whose role is just 
to store data on disk.  So you 
can either do this in shared 
preferences, or in the database.
Next, we would have the 
repository whose role is to 
fetch and store data.  And, 
optionally, it could also do 
in-memory cache.  So the 
repository will be the class 
that mediates between the local 
and the remote data source.
Because the business logic was 
quite complex, we decided to add
another layer: Use cases.  So 
the role of the use cases is 
just to process databased on 
business logic.  This will be 
small, lightweight classes that 
can be reused.  So the use cases
depend on repositories and/or 
other use cases.
And next, we would have a 
ViewModel.  So the ViewModel's 
role is to expose data to be 
displayed by the UI, and also to
trigger actions based on the 
user's actions.
And the ViewModel would depend 
on use cases.
As an input, the ViewModel would
give IDs.  It would give IDs in 
the case where it is a model for
a detail screen, for example, 
and it would give the user's 
actions an an input.  And 
output, it would return a 
LiveData.
And next in the UI, we would 
work with activities in XML.  So
the role of these is just to 
display the data and to forward 
actions to the ViewModel.
As an input, they would get the 
optional ID and the user's 
actions.
So we looked at our application,
or the architecture, and we 
divided it into three layers: 
The data, domain, and UI.  When 
we decided to be more 
opinionated in the way that we 
use the libraries we are using. 
We knew that the LiveData really
shines when it is used with an 
activity or a fragment.  We 
decided to really keep the 
LiveData only between the 
ViewModel and the UI, and that's
it.
And even more, was of the nice 
integration between LiveData and
data binding, we decided to use 
data binding in our XMLs.
But again, still with all of 
these constraints and all of 
these guidelines that we've set,
there are so many ways in which 
we can actually implement this 
kind of architecture.  And we 
knew that Kotlin and the Kotlin 
language features will help us 
improve this even more.  And 
more precisely, what we 
particularly like are the 
functional constructs that 
Kotlin supports.  And one of the
first decisions that we have to 
make is to handle asynchronous 
operations.  And we decided to 
work with coroutines as the 
backbone of our app, with 
coroutines, it is easy to launch
a Coroutine and handle the 
response.  And more precisely, 
what we liked is the fact that 
coroutines have a scope.  So, 
for example, let's say that you 
are opening the activity, you 
are triggering a network 
request, you want to make sure 
that when you are pressing back 
and exiting that activity, you 
are also cancelling that network
request.
So this scoping of it the 
coroutines is something that we 
liked.
So this meant that we decided 
that, in the ViewModel, would be
the place where we are launching
and we are cancelling coroutines
and we are also making that 
transition between coroutines 
and LiveData.
But then, for all the other 
layers above the ViewModel, we 
would use suspension functions.
And these suspension functions 
would return a result class.
So, more precisely, this result 
will have two types: Success or 
error.
And this is because we wanted to
make sure we are not throwing 
exceptions here and, there but 
that the errors represent a 
state.  So what is interesting 
in Kotlin is that if you want to
be able to ectend the class, you
have to market  mark it as open.
This means that classes are 
final by default and you have to
be intentional when using 
inheritance.  That means that 
Kotlin supports the idea, the 
best practice of favoring 
composition versus inheritance. 
But we can do better than using 
open classes.  We can use a 
sealed class, because with a 
sealed class, we can restrict 
the class hierarchies.  That 
means we cannot extend the class
outside this file.
So a lot of times when we would 
use this result class, we would 
typically use it
inside a with, it has smart 
casts, when there's a success, 
you do something, when it is an 
error, you do something else.
But every time we were using it,
we wanted to make sure that we 
were always handing every case 
and making sure that by mistake 
we are not handling something, 
we wanted the compiler to tell 
us that, hey, you forgot 
something, you forgot to handle 
the error case.  This meant that
the when needs to be exhaustive.
But when is exhaustive only when
used as an expression.
So we added, we created the 
exhaustive property.  So more 
precisely, we created an 
extension property where we are 
returning the object.
Here's another problem that we 
have.
So we have a comment class and a
comment with replies.  So the 
difference between these two, 
and the fact that the comment 
also holds the information about
the user that posted the 
comment.  So it will have the 
display name and the portrait 
url, and the comment with 
replies is pretty much a tree 
structure that holds the replies
of the comment and the replies 
of the replies and such.  What 
we had to do was to build a 
comment out of the comment with 
replies and a user object.
So you said, okay, that is easy,
create a new constructor that 
gets the parameter, the user, 
and the comment with replies and
that is it.  But the thing is, 
we didn't really like this 
because the classes were in two 
different layers, why should the
comment know about the comment 
with replies, why shouldn't it 
know about the user? Why should 
we need to changes this.htm? So 
what weep ended up using is an 
extension function.  So more 
preciseprecisely, we built an 
extension function to the 
comment with replies, and that 
will create a comment.  So when 
you are building an extension 
function, you only have access 
to the public fields.  So this 
means that we're not, by 
mistake, accessing or changing 
any private implementation and 
it allows us to keep our classes
focused on what they do without 
extending them.
And so it meant that we didn't 
have to college the public API 
and we would avoid accessing 
private implementation details.
So what I like about data 
classes is the fact that they 
are value objects, this shines 
when used in tests.  So for 
example, we had an out vote flag
in the comment.  So when we 
checked the test to see if it 
was up voted, it would have it 
to false, would upload the 
comment and check whether the 
expected result is similar to 
the initial comment, but with 
that up voted flag to true.  And
because the comment has so many 
fields, it is easy to make 
mistakes and what is actually 
important here, the fact that 
the up voted flag has changed.  
With Kotlin, you can use the 
copy method.  We create a copy 
of the orbital and -- that it is
called on, and we are setting 
the flag, the flag that we are 
actually changing.
And that's it.  The code is more
concise and more readable, more 
comprehensible.
So let's take another example.  
So we had another, in the app, 
we are working with a remote 
data source to post a comment.  
And here, we would expose the 
suspension function that return 
a result.  And inside this 
method, we would create a new 
comment request, which would 
trigger that request in the 
back-end, we would await for the
response, and then we would 
handle the response, building 
the results success, or the 
result  result of error, 
depending on what is needed.  If
you look at this code, this is 
actually not enough.  In the 
case when your device is 
offline, this code will crash.  
So what we had to do is to run 
up every request inside a 
tricatch, and we had a lot of 
requests.  So we saw that we 
keep on adding and adding this 
tricatch everywhere, and then 
our methods were loaded so we 
couldn't really focus on what 
really mattered on building the 
response, triggering the 
request.
So what we did is create a 
top-level function.  So this 
would  would be a suspending 
function with a Lambda or the 
error message.  Here, it would 
trigger the code, wrap it inside
a tricatch, in the case of an 
exception or result of error, 
based on the error we passed in.
So this means that in the null 
data source, we could just 
create a safe API call and then 
put the call that matters for us
inside another function.
The code became more readable 
and easy to understand.
So this safe API call, I was 
saying that it has the call as a
first parameter and the error 
message as a second one.  In 
Kotlin, if you -- if the last 
parameter of a method is a lam 
it Lambda, it means that you can
use it as a trailing Lambda.  
That meant that when you are 
calling this, instead of passing
these two parameters, we can 
just pass the error members of 
the jury as a parameter of the 
function and then use the 
trailing syntax to call the 
method.
So like this, the code becomes 
more concise.  But it is more 
readable, so when we looked at 
this, it felt like what matters 
here the most is the error 
message, which is not really
the case.  What mattered is we 
get the first comment.  So 
although the code is more 
concise, it doesn't mean more 
readable.  So brevity is not 
necessarily a good thing.  If 
Kotlin offers all of these 
features, be mindful and think 
of whether you need all of 
these, and use them in the right
places.  This is another 
example.
So, as soon as we were switching
to Kotlin, in the activities, 
the first thing that we did is 
make all of the views latant, 
becausewy  we didn't want all of
the nullability.  We looked at 
the code and saw that we 
shouldn't do this.  Some views 
are no result views, or only 
inflated when specific 
conditions are met.
So actually nullability was 
good, nullability can be 
meaningful.  Nullability was 
telling us that something is 
missing and that we should 
really handle it.
So,
overall, all of these features 
from Kotlin, like coroutines can
help us shape our app.  And 
together, with the guide to app 
architecture, help us build this
maintainable, this safer and 
faster to develop architecture 
that we wanted to have.
Thank you.
[ Applause ] 
1Y50IRK9SDS
[ Applause ] 
Coming up next: Getting the Most
from the New Multi-Camera API by
Emilie Roberts, Oscar Wahltinez,
Vinit Modi  . 
SPEAKER: Hi, everyone, welcome 
to this session on the new 
multi-camera
API.  I'm vinit ModModi. 
I will give an update on the 
state of camera.  Most camera 
apps focus on the native camera 
app that ships with the device. 
More than twice the amount of 
usage occurs on the apps that 
you build.  And it is extremely 
important that you support the
new features that are available 
in the new
Android APIs.  When we speak to 
a lot of developers, the number 
one question is the state of 
camera 2 API and where we are 
going forward.  Starting with 
Android P, you will find that 
almost new devices will support
camera 2 and HALv3.  You will 
see that the characteristics are
unlimited, like a point and 
shoot, a full, with advanced 
capabilities, and level 3, with 
YUV processing and raw.
We are working with several 
manufacturers to open new APIs 
at launch.
We're excited that the pixel 3 
and the HAUWEI20 series support 
the API.
Why is this so important? Prior
to Android P, as developers, you
get access to one of the center
sensors and the native app the 
whole capability.  We have the 
native camera app, with the 
sensor and the logical camera, 
an abstraction of all of the 
physical sensors that allow you 
to take advantage of the 
hardware.
There are several use cases and 
possibilities with the new 
multi-camera API.  Today, Oscar 
is going to talk about optical 
zoom and Emilie is going to 
cover bouquet. 
SPEAKER: I'm in thethe. 
SPEAKER: I'm on the developer 
relations team, and we will 
start with the live
demo.  What could go wrong.  Are
we on?
Nope.
Here we are implementing the 
multicamera zoom.  We are 
swapping, at the UI layer, the 
two camera streams and not doing
any kind of zoom or cropping.  
And you can see it is 
instantaneous, and this is just 
a single session.
And I'm swapping the two camera 
streams.
And I have the same thing here 
on the pixel phone, and the main
idea is that you can see it is 
working on -- if it works
again.
Had this been working, you would
see the same demo.
[ Laughter ].
In any case.  We will go back to
the
same slides.
The idea is that the single 
camera session, to the streams, 
we are going to swap between the
streamsstreams and show you how 
it was built.  We have the same 
running on both devices, a feat 
to have them on both devices 
something that is tied to the 
hardware and the camera.
So we will talk about how we can
use multiple cameras 
simultaneously, the basic 
guarantee provided by the API is
that you can use two streams at 
the same time.
Recall the guaranteed stream 
configurations for single camera
devices.  It is a set of rules 
based on higher level, target 
type, and size.  If we use the 
multi-camera APIs correctly, we 
can get an exception to these 
rules.
We will illustrate it with an 
example.  We have a single YUV 
stream of maximum size.  As to 
the previous table, devices with
minimum hardware level will be 
able to use a single stream 
without configuration.  If we 
use the multi-camera APIs, we 
can use two streams of 
equivalent configuration from 
the underlying physical
cameras.
Let's walk through what we are 
going to do to implement the app
we demoed earlier.  We broke it 
into five steps.  Are you ready?
Step number one, find the 
physical cameras.  We start by 
defining pairs of physical
cameras that can be operated 
simultaneously.
Using the subject, we look for 
test capabilities.  If logical 
multicamera is one of them, we 
know this device is a logical 
camera.  And now that we found a
logical camera, we store it, we 
will need the ID for later, we 
will see.  We get the physical 
cameras associated with it.
And then we can move on to the 
next step.
Here are some steps for what I 
just described.  You take the 
logical camera, you call get 
physical camera IDs, and we 
create the physical cameras 
associated with the eligible 
camera group.
On to the next step: Open 
logical camera.
The second step is nothing new, 
open the camera.  Recall the 
camera ID we set earlier, that 
is the only one we use to pass 
to the camera manager.
To reiterate, we only open the 
logical camera.  This triggers 
whether the device is ready.
We have now opened logical 
camera.
In the next step, we will create
the applicable configuration 
updates.  They are used to 
create the camera session.  For 
each desired target, we may have
a physical camera from the list 
we found earlier if we want to 
retrieve frames from a specific 
camera.  We will go into more 
details.  We create the output 
configuration object, using the 
desired output target.  If we 
want to associate that output as
a specific physical camera, then
we pass the AD in the set 
physical camera ID API.  If
we want to use eligible camera, 
we can skip this step.  We also 
may is have a configuration of 
both.
So, at the end of the day, we 
have a list of configurations, 
some of which are associated 
with physical cameras, and some 
of which logical camera.  The 
goal is to put all of the 
configurations into a single 
session  session configuration. 
Each configuration as an output 
target and a physical camera ID.
Now we create the capture 
session.
How do we create the capture 
session using the new session 
configuration object?
We start off with our list of 
output configurations that we 
just created.
With that, we instantiate 
assessing configuration, which 
includes the capture session 
callback.
From that callback, we are going
to give the instance of the 
creative camera session.
We take that session 
configuration object, and the 
camera device, which we got from
a step number two when we opened
the logical camera, and we send 
that framework to the request to
create a new session with the 
desired configuration.
The callback provided in the 
session configuration object 
will be triggered now, and then 
we will have our camera session 
ready to be used.
Last step, capture
requests.
Once that has happened, we can 
start getting frames out of the 
cameras.
For example, you want to capture
a frame from two physical 
cameras simultaneously.  We take
the session we created earlier 
and the pair of output targets, 
and each is associated with a 
specific camera ID.
We create a capture request that
we don't normally do, in this 
case, we send to output preview.
We attach the output targets to 
it that we normally do.  And now
we dispatch the capture request,
nothing different here, except 
in this case the output services
will receive image data from 
each of the associated physical 
cameras and the capture request 
callback will trigger only once.
So again, it is just like any 
capture request, the big 
difference is that the 
completion callback will give me
back two start exposure time 
stamps instead of just a single 
value from normal capture 
requests.
So to recap, this is how I 
implemented the optical zoom 
demo.  We found the physical 
cameras, we opened the logical 
camera that is part of the 
group, we created the output 
configurations, we planted the 
list, created the capture 
session, and then we distached 
the capture
requests.
One more topic I wanted to touch
on is lens distortion.
A classical example of lens 
distortion is the fish eye lens,
this is for illustration 
purposes only.
All lenses have some amount of 
disdistortion.  For logical 
cameras, that is minimal and 
corrected by the drivers.  For 
physical cameras, the distortion
can be significant.  The 
physical lens distortion is 
described in a set of radial and
tangential coefficients, using 
the lends distortion key.  The 
commentation has a lot more 
details if you are interested.
The good news is that
there's a way to correct 
distortion without doing a lot 
of math, you can set the 
correction mode on a capture 
request.  And off means that no 
distortion is applied.  
We may need to use this if you 
want to do things like 
synchronization, we may touch 
oon it later.  And fast meanses 
the corrections are applied 
while meeting the target frame 
rate.
High quality means that the 
distortion is corrected as much 
as it allows, potentially at the
cost of frame rate.  If we do 
not specify a correction mode, 
it is fast or high quality, to 
the implementation details which
is default.  You can see which 
is applied to your capture 
request.  We will demonstrate 
how this is set to high quality,
probably what we want for a 
still ImageCapture.
Assuming that we are restoring 
the camera captureception, we 
instang  
instantiate it.  And then we 
determine if the high quality 
distortion mode is available.
And now that we know that we 
have high-quality correction for
the distortion, we settled on 
the capture request, and we do 
what we always do, pass the 
capture
request.
For more sample code and 
caconical details, take a look 
at the blog post.  We covered 
this and some more, we posted 
earlier this week, and now we 
will hand it over to Emilie. 
SPEAKER: Thanks, Oscar. 
I'm emilie Roberts, and I will 
use this on
the bokeh effect.  And the first
has no camera at all, I will 
show the mechanisms for creating
the bokeh effect.  When you get 
into the dual cam, we can focus 
on the multicamera aspects and 
not worry about the bokeh 
effect.  It will be published, 
open source, don't worry about 
scribbling too much code.  So 
can we go to this
phone?
Okay, so we have -- I didn't set
this up properly.  We will do 
the single cam bokeh effect, you
can see on the screen, cutting 
me out, bump up the final 
result, it is pasting the 
portrait mode in there.  This is
a rough cut portrait mode. And I
have an optimization on this, 
let's see how that goes.
I will show you the output steps
here.
So it is trying to do a better 
job of finding the fore ground, 
it didn't do too bad.  It is 
generating the fore ground 
image, the background, which is 
just 
let us down, pasting on the 
final result.
So that is not too bad for a 
single cam.
Let's try the dual cam demo.  
With these stage lights, I'm not
sure.  C'mon.
Hey, not too bad.  We are doing 
good.
So you can see a depth map was 
created in the bottom left-hand 
corner that is detecting me in 
the fore ground, and the rest of
you are a little bit faded out, 
you can see that closer folks 
are in gray and the black to the
back.  You can see the lights 
wreaking havoc, I will show you 
the final result.  There's a few
optimizations that can happen.  
But it is working pretty well.  
So again, this is using the two 
front cameras on the pixel two. 
You can see the two streams 
going at 
once, oops.
Will this connect back up? Both 
streams at once, the wide and 
the normal angle going at the 
same time.  Can we head to the 
slides, please.  Let's talk 
about how we do that, there we 
are.  So we had the normal 
camera and the wide angle 
running at the same time.  We're
going to publish this probably 
on GitHub, open source, so you 
can dig into it and help 
optimize it and make it better. 
So the single cam, the floating 
head bokeh effect I call it.  
We're going to take a photo, 
face detect, two copies, we have
background, fore ground, do 
fancy background effects, and 
paste the floating head back 
where it belongs.  Face detect 
is built into the camera 2 API, 
easy to implement.  We will 
check the camera characteristics
to see if the camera supports 
face detect.  If it does, you 
find the mode you want.  There 
is off, and then simple, and 
full, depending on your camera 
device.  And then when we make 
our camera capture request, we 
will include that in the 
request.  When you get our 
results, you can see if the mode
was set.  If you found any 
faces, in this example, I 
searched for the first face that
it finds is the one that I use. 
We can use this to expand 
multiple faces.  And face detect
grabs the face.  So I bumped up 
the bounds a little bit so it is
more of a head getting chopped 
off, that sounds bad, a head 
pasted on to the background.
And let's talk about the fun 
background effects.
You can do what you want, using 
render script, we did a blur.  
Because it is a multi-camera 
talk, some have a zoom, you can 
do a background with another 
camera and zoom it out of focus 
so you can do an optical blur 
and save you the software step.
And in this demo, we did a 
custom software sepia effect 
using render script.  If you are
using multicam, a lot of cams 
have built in effects, like mono
chrome that you can query and 
capture.  If you are not used 
render script, it looks like 
this.  For the blur effect, we 
care about the three middle 
lines and there's a built-in 
script intrinsic blur, pretty 
handy, and it basically works 
outside of the box.
In this case, it blurred outside
of the box because the box is 
not blurry, ha.  This is a 
custom render script script for 
the sepia effect.  You can see 
in the first three lines, we are
taking the input red, green, and
blue channels, muting the 
colors, making them yellow, and 
setting those to the output 
channels.
Okay, so we have the background,
a cool effect on it, what do we 
do with the fore ground? We have
the face cut out and we apply 
the border dev with the linear 
gradient so when we paste it, 
there is not the harsh line.  
And ta-da, we paste it on, 
things look pretty good.
There's a couple of 
optimizations.  One you saw, 
which is with the grab Cut 
algorithm, built into open CV, 
the computer vision library for 
the depth map demo later on.  
And basically, I found the -- I 
chose a rectangle a bit lang  
larger to guess where the body 
might be, and then you have the 
magic wand tool in the photo 
editor to shrink it down.  And 
then we can add in multiple 
faces.
  Now, the moment you have all 
been waiting for, we will talk 
about dual cam bokeh with the 
depth map.  We're going to use 
two cameras simultaneously, and 
we're going to create a depth 
mapmap, the hard part, which is 
why I highlighted that in bold. 
We will use the same mechanism 
we talked about.  How does this 
work? First of all, the double 
capture.  So this, on the left, 
is me hanging out with my pets. 
The left is the normal camera, 
the pixel 3, and the right is 
the wide-angle shot.
To do that, just as Oscar one 
walked through, we set up 
multiple output configurations. 
For each lens, we set up, here, 
we have the preview surface and 
the image reader for the normal 
lens.  We use set physical 
camera ID to the normal lens and
we do the same thing for the 
wide angle lens.  So we end up 
with four output configurations 
we are putting into our con
figeration.  From there, it is a
matter of choosing output 
targets for the capture.  We 
want the photos, so we want the 
image reader from the normal or 
wide-angle lens.
Okay, so we have our images, and
now we have to do a bunch of 
math and magic and make that 
boket effect happen.  I want to 
give a brief introduction to 
stereo vision before we get into
the code.  Looking at these 
slighted  slides y , I got 
bored.  I like geometry, and I 
started asking, what does P 
stand for, anyway? Obviously it 
is a pile of chocolate.  P 
stands for pile of chocolate, 
this is what we are focusing on 
for the rest of the demo.  And 
camera one is a little bit 
boring, camera two.  So as here,
we are going to replace with a 
character, this is my friend, 
pepper the shark, and this is 
hippo, that are going to help us
talk about stereo vision.  So 
left camera normal lens is 
pepper the shark, and right is 
Suzie lou, the hippopotamus.  
And they are zeroing in on that 
chocolate.  And the skewed 
rectangles there, the 2D 
surface, the image that the 
cameras are going to capture.  
The 2D representation of the 
real-live 3D object we have.  
Let's look at what that look 
like.  The shark eye view is on 
the almond, and the hippo is on 
the crunch.  They have this 2D 
representation and we will take 
their separate views and combine
them so we get a little more 
information than the 2D view and
create a great depth map.  So we
have that, again, the normal 
view, the wide-angle view, in 
this case, they are both normal.
The left and the right hand, the
overlay on each other, and you 
get that 3D ruler effect from 
elementary school that you hope 
you got to enjoy as a child.
And, from there, we can create a
depth map, which allows you to 
do really cool things, like 
awesome bouket effects, know how
far away the chocolate is so you
can reach out and grab it.
Okay, so those two cameras, 
those two pictures are at a 
different orientation from each 
other and they are separated in 
space.  So we need to get those 
on top of each other.  This is 
what we call the camera 
extrinsics, how the cameras 
relate to each other.  So we 
need to rotate and translate the
images so they appear on top of 
each other.  Normally we say -- 
we normally give the rotation 
and translation parameters in 
camera in relation to world.  So
instead of camera one to world, 
we will have shark to world, and
hippo to world.  But when we are
doing stereo vision, we need to 
worry about shark to hippo.  How
are the cameras related to each 
other.  Like a good engineer, I 
need to switch helpo hippo to 
world and I have a pathway, 
shark to world to hippo.
You can see the math on 
Wikipedia, and it looks 
something like this.  To get the
rotation matrix, we will reverse
it for camera two and 
cross-multiply it with camera 
one.  And we will take the inner
product and subtract.
You can read all about it on 
Wikipedia or other sources.  If 
you are working on this 
yourself, work on the 
translation matrix from pixel 3 
from the normal to the wide 
camera.  What do you notice 
about this? The 9 millimeter 
separation between the cameras 
looks just about right.  If you 
look at the phone, you know 
there's a good, what is the 
American, a good -- anyway.  
There's a good nine millimeters 
between those cameras, that 
makes sense.  What I didn't 
notice, it cost me about a week 
of time, it is in the Y 
coordinates.  The cameras are on
top of each other.  So while I'm
working on this phone, looking 
at the cameras beside each 
other, I assumed that they are 
horizontally displayed, but it 
assumes that they are right next
to each other.  And camera 
sensors are often optimized for 
landscape, which makes sense.  
If you do it wrong, your depth 
maps don't work, you pull your 
hair out and have a great week 
like I did.  Just a note when 
implementing this.  So you have 
the camera extrinsics, how to 
get the pictures from the camera
on top of each other.  
Intrinsics are properties of the
cameras themselves.  We have a 
normal and a wide angle lens and
they have different properties. 
So there are two things, the 
camera characteristics, the 
focal length, the principal 
access, if that is skewed, this 
appears often in the three by 
three matrix, and distortion.  
The lens, you are going to get a
little bit of distortion going 
on that we need to consider as 
we are mapping the two images 
together.  And another note, we 
will use the distortion 
properties of the lens.  But the
camera undistorted it for
you.  
We will undistort it and 
redistort it, turn it off if you
want to do depth maps.  That is 
easy for the request, make sure 
the distortion mode is off.  
Okay, here are the four things. 
Rotation, translation, the 
camera characteristics matrix 
and the lens distortion.  How do
you get these properties? Easy. 
You take an afternoon, print out
a checker board, has anyone done
this before? It is fun.
Camera calibration.  Take a 
series of shots with both the 
cameras, run a bunch of 
algorithms, figure out these 
four camera characteristics and 
then you can start making depth 
maps from the cameras.  You can 
tell from my cheerful face that 
it is not that fun, don't do it,
it is no good.  Luckily in the 
camera 2 multicamera APIs, we 
have these great fields: 
Rotation, calibration, 
translation, and distortion.  
You can get it out the API which
is wonderful.  If you are 
implementing this yourself, so 
the camera characteristics, the 
focal length, and the access 
information comes in five 
parameters.  This is in the 
Android documentation.  But to 
create that three by three 
matrix, you have to follow the 
documentation and plugin the 
numbers.
Another thing that might throw 
you off are the distortion 
coefficients, the five values, 
and the open CB value library 
uses them differently than out 
of the API.  You need to 
remember it is 0-1, 342.  If you
use them in the 01234 order, 
when you undistort your images, 
they lack like look like they 
are in a whirl pool.  So there 
is something wrong with the 
coefficients.
  After the parameters, we 
prepare the images to do a depth
map comparison.  This is me in 
my kitchen, yes or no if I don't
know if you can see, but from 
the ceiling, there's a curve 
going down, the distortion 
effects we are talking about 
with the distortion correction 
off.  And when you are comparing
two images, the straight and 
curved lines need to line up.  
We call that rectifying and we 
use the camera characteristicses
characteristicsesx to do that.  
All of the functions are in the 
open CV library.  The first is 
stereo rectify, it gives a set 
of parameters to define these 
calculations.  We pass in the 
values we got from the API, the 
camera matrix, the distortion 
coefficients, the rotation and 
translation that we calculated 
before.  We get these parameters
out, we call undistort rectify 
map, which gives us a map 
telling us how we can take two 
images from these cameras and 
map them to each other.  And the
remap function does just this.  
So lets see what that gives us. 
.  On the left again, from the 
normal cam, front cam of the  
the pixel three and the wide 
angle lens.  You can see that 
the lines are lined up, the crop
is right, the roof lines, the 
door lines are straight, there 
is no whacky distortion, from 
where you are sitting, you 
probably have to look closely to
notice that the left-hand 
picture is a little bit closer 
to the left-hand side of the 
frame.  So there are actually --
they are offset by a little bit 
which is what you would expect 
if you had two cameras nine 
millimeters apart.
So we have the images, we have 
undistorted it and rectified 
them and we are close to 
creating the depth maps, we call
the depth map function.  We use 
stereo BM, or stereo SGBM.  One 
has a few more parameters than 
the other.  When you play with 
the open source demo, you can 
see how these parameters work 
and play around with it.
Optimize them.
Commit your changes.
Help make that better.
We call compute and make the 
step map.  When you do this, you
get an amazing photo, something 
like this.
And actually, sometimes it looks
a lot better than that.  This is
not quite what we want to work 
with.  We want to filter that, 
in this case, using the weighted
least squares filter, that 
smooths it out and gives more 
useful depth map.  So the darker
pixels are farther back, and the
whiter are the closer ones.  It 
is hard to see, you can see the 
shark's snout and the hippo's 
snout are a little bit grayed 
out, so it is actually working 
to some extent there.  This is 
what we call the filter, it is 
also included in the open CV 
libraries in the contributor 
modules.  So it is all open 
source.
And it is really cool when you 
get a depth map that
isS per feck, it is 
exhilarating.  So here we have 
the depth map.  
What do we do with it? We can 
apply it as a mask on top of it,
and the black areas we want to 
fade out and we want to 
highlight the fore ground.
That is pretty easy to do with a
porter
def, and the result is something
like this.
So, Indeed, the fore ground is 
more present and the background 
is faded out.  I have high 
standards, I see a translucent 
floating shark over my shoulder,
my face is grayed out, my eye 
ball is missing, I will put a 
red X through it, not quite good
enough, but we want a depth map 
more like this.  We will put a 
hard threshold on it and decide,
fore ground, background, that is
it.  In either apps, you may 
want to do something similar, 
but it could be a smoother 
curve.  To do that, we can use 
the open CV function threshold.
We give it some cutoff value for
the app, it is somewhere around 
80 to 140 out of 255, and that 
is just that limit where 
something is considered fore 
ground or background.  I wanted 
to note this in case you are 
implementing any of this.  When 
we applied the mask, you need to
turn the black pixles to
transparent, this converts all 
of them to
transparency.  We are almost 
there.  In the middle picture, 
my eye is a bit blacked out.  
Just remember that for three 
more slides or so.
So we have the initial picture, 
we have the depth map, we do 
this hard threshold on it, and 
we can again create our 
background just like we did in 
the first demo, blur it out, 
mono, monoChrome it, and cut out
that fore ground.  We have all 
the pieces we need to paste it 
on.  This is our amazing, final 
portrait shot.
Which -- that is pretty good.  
I'm proud of it.
So we will talk about an 
optimization.  And remember the 
eye ball thing I was talking 
about? So anything kind of 
gleaming and shining can get 
messed up in this current 
iteration of the application, or
bright lights can throw off the 
depth map.  I did one 
optimization, we have the face 
detect region.  I'm pretty sure 
I want the face in the fore 
ground.  So I used it and cut it
out and said anything in the 
face is going to be the fore 
ground.  So that protects it, 
like my teeth and my eye from 
the masking out effect.
I don't know if you noticed, we 
will go back, my fuzzy red hair 
and the red couch, it kind of 
blended in.  So we can use grab 
cut to do a little bit better 
job of figuring out exactly what
is in the fore ground.
So thanks a lot, we really hope 
that this gave you a deep dive 
into using camera 2 and the 
multi-camera APIs, and gave you 
exciteg and creative ideas.  We 
want to hear your ideas, see 
them in the apps, and know what 
features you are looking for.  
We think they are great and we 
want to keep pushing the camera 
ecosystem forward and keep doing
more and more stuff 
ecosystem-wide.
Thanks so much again and please 
do come to the camera sandbox if
you want to ask us any 
questions, follow-ups, try this 
app, see if it works, look 
forward soon to being open 
source.  Thanks a lot.
[ Applause ].
 SPEAKER: Now that leads us to 
our final snack time of the 
summit.  We have coffee and 
snacks and coffee and fruit and 
cheese and stuff like that for 
you all.  But a few things I 
want said ed to call your 
attention to, please rate the 
sessions so we have a QR code 
that will come up here 
momentarily, they are 
incredible.  The team is just 
amazing.  So this is how you 
tell us how good of a job we 
have been doing.  Let us know 
what you think of our speakers, 
our sessions, one other thing.  
If youmissed sessions, all 
sessions from yesterday are 
online, and by the end of today,
every single session from today 
is going to be online.  So you 
can go home and binge watch all 
of the things you didn't get to 
see.
And, finally, come back and make
sure to come back.  We have 
great sessions coming on later, 
and in both rooms, we are not 
quite done yet.  And enjoy your 
snack break.
[ Applause ] 
SPEAKER: [ Applause ]. 
>>> Hello, everyone.  Welcome to
the deep dive into Android 
Studio profilers session.  I'm 
David Herman SPEAKER: I'm 
Shukang Zhou.  We are developers
in the Android Studio. 
SPEAKER: We wanted to guv you an
outline of what to expect in 
this talk.  Instead of a high 
overview, we will narrowly focus
on a few features that we think 
can help you get a better handle
on any code base.  We will drop 
some tips and tricks along the 
way.  We are going to be 
profiling a real app, Santa 
Tracker.  It is an app by Google
which allows users to track 
Santa on his course around the 
world on Christmas Eve.  The app
also contains games and a couple
of other extras.  We are -- they
release a new app every year.  
We're using the one that is 
publicly available on GitHub.
Finally, I want to mention two 
talks about profilers that are 
previously given this year, one 
at Google I/O that did talk 
about profilers at a higher lef,
also introducing what is new in 
Android Studio o3.2, and the 
other at the Android gamedev 
summit about profiling your game
.  That talk focuses on 
performance, native code, 
related tools, things like that.
You can find those vade  -- 
videos on YouTube.  If you are 
watching this online, we 
included the links in the 
description below.  If this is 
your first time coming and 
learning about profilers, you 
are just curious, here's a quick
overview.
The studio profilers feature is 
divided into four main 
profilers, one for CPU, one for 
memory, one for network, and one
for battery.
There's also an event profiler 
that is on the top of -- that is
always on the top, which lets 
you see things like user events,
such as taps, keyboard events, 
screen rotations, and life cycle
events.  So when your fragments 
and activities start and stop.  
That's enough talking, let's 
jump into the demo. 
SPEAKER: Okay.
Let's start the demo.
So first, we want to click on 
this button.  I can launch the 
app, and I will start the 
profilers.
And so today, we are using 
Android Studio 3.4, canary 3, 
because that's the latest 
version of what we have.
Since the canary release is not 
a stable release yet, so there 
might be some bugs, please bear 
with us if anything interesting 
happens today.  The app we are 
using today, the Santa tracker, 
contains only Java code.  So the
Java apps will be the focus of 
today's talk.
So, as you can see, in the we 
have four
profilers: CPU and energy.  You 
can click on any one of them to 
get more data.  
We will jump to the CPU 
profiler.  Here, as you can see,
in the CPU profiler, you can see
the CPU utilization and the 
thread state.  They are useful 
to tell you when your app 
becomes CPU bundled.
And if you are examining two 
related threads, one trick is 
here, you can reorder the 
threads any way
you want.
To know further data information
about which path of your code is
executing or how they are 
executing, you need CPU 
recordings.
And let me get a little bit more
space.  So, as you can see here,
there are four types of CPU 
recordings.  So we will go to 
the first one, called Java 
methods.  We will get a quick 
recording of this one.
So during this type of CPU 
recording, the Java virtual 
machine is theoretically we 
collect the core stacks of all 
the Java threads in your 
process.
And then it will present the 
stacks in the -- in this part, 
which we called the recording 
details -- the details of the 
recording.
And, after the recording is 
done, the entire recording is 
automatically selected and if 
unit to  -- you want to take a 
closer look, there's a button 
called zoom tool selection.
So if I click on it, it will 
fill the entire screen.  And if 
you want to just see a sub range
of the recording, you can select
using the mouse.  If you want to
select the entire recording, you
can click on this small clock 
icon here.
And,
a very specific point.  In that 
case, you can do a single click 
in this area and you can 
automatically select something 
here.
Now let's take a look at the 
call stacks here, and let's 
select this range that might be 
more interesting.
So, as you can see, the profiler
will color the cost from the 
Android platform in orange, the 
method from the Java language in
plu,- blue, and it will color 
any of your code in the library 
to green.  So if you want to 
know what method from your code 
is running, you will be looking 
for the green stuff.  So here we
see some green stuff here and 
here, you can see, this is an 
on-call method from the village 
view class.  And if it is not --
if you see the code base, you 
can easily see the shield is 
responsible to draw the cloud 
that is on this 
screen.
So another thing I want to talk 
about the sampling is you can do
some justm customization about 
the recording time.  You can 
click the edit configuration
entry, you can create the 
customized CPU configuration.  
For this Java method recording 
type,
rue can change the inter  
intervales.  The more quickly 
you select samples, the more 
quickly the data will be 
represent said ed to you.  That 
will incur more overicide. Head.
Depending on the use case, you 
want several tries until you 
find the sweet shot.  As we have
seen, the sample Java method is 
very useful to get a very 
high-level picture of which part
of the code it is running.  In 
some other scenarios, if you 
want to focus on a smaller area,
the second type of the 
recording, trace Java method, 
will be more interesting.
So let
me collect another trace.
So in this type of CPU 
recording, the Java virtual 
machine is selecting the  -- 
collecting the data every time 
it executes a method.  So 
there's a lot more data being 
collected.
And so I wanted to understand 
what method from my code is 
running, then you wanted to look
for the screen stuff here.  You 
can see that there's a lot of 
things, you want to zoom in.  
You can use the WAS keys to zoom
in.  So the W is the zoom in, 
you can see with the key I'm 
pressing.  The S is zoom out, 
the D is moving to the right, 
and A is moving to the left.
So if you are gamer, you 
probably know this.
So if you zoom in, let's see 
what are the green stuff.
So a big one is the one that we 
just saw observe, the methods 
from the village view.  And they
are responsible for drawing the 
clouds.  And here is the method 
from the snowflake view class, 
they are responsible to draw all
of the snow flakes that are 
floating around.  If you look 
closely, there are many tiny 
green lines here.
So you can further zoom in, you 
can further zoom in, you can see
-- I will zoom in.
So you can see that this is the 
method from the snowflake class,
and this is also from the snow 
flake class, this is also from 
the snow flake class.  So let
let's -- you may
want to see what data  source 
this goes.  So you can jump to 
the source code.
You can see in the snow flake 
class, on this method, we are 
doing some calculation about the
velocity about the angle, about 
the size, and then we draw and 
it is very clear that every line
of this screen will execute the 
method.  That is why we are 
seeing so many calls into this 
method.
So ass as
So ass as you can se you can 
see, you can see what is 
executed and how often it is 
executed.
And another thing I want to talk
about here is the chats here.  
The first is the call chat, that
is representing all of the call 
stacks during the CPU recording.
So it is -- the things to the 
left, well, we will select the 
entire range so you can see.
So the call stacks on the left, 
it is what happens first in this
recording.  So the things to the
right, they will happen later in
this recording.  In the frame 
chart, it is similar to the call
stack, but it is upside down.  
So the root is on the bottom, 
and also, identify call stacks 
aggregated here.
So it is very easy to see the 
total time the method has been 
executed.  The
top-down, it has exactly the 
same data as the frame chart.  
It is just represented in a 
different view.
What is nice about this view is 
you can sort these methods by 
the time.  So the sales time is 
the sime executing by this 
method itself, the children is 
executed by the sub routines 
called by this method, and the 
total time is a combination of 
the two.
Bottom-up is looking at the call
stack upwards to the caller, 
basically from the method to 
another 
function, to that call. 
So you can see it is very useful
to see where this method is 
called and how much time this 
method has executed when it is 
called by that specific color.
And there is a third type of CPU
recording that is called related
functions, C or C++ functions.  
We will do another trace here.  
And if you remember, I just said
that this app, the Santa 
tracker, it contains Java, only 
code.  So therefore the call 
stacks, assuming it is corrected
by this type of recording, it is
not very interesting.  They are 
mostly the system calls, and 
some of them are in the Android 
framework native code.  If your 
app has any native components, 
this type of recording will be 
very handy.
There is another type of CPU 
recording called trace system 
cost, and before I go into the 
details of that, I will head 
back the
demo back to Dave. 
SPEAKER: CPU recordings are very
useful.  Sometimes there's an 
exact function, or maybe a 
couple ofphic  functions you 
want to analyze and it is a 
little bit imprecise to record, 
do something in the app, stop 
recording and zoom and search 
for it.  We provide a very 
simple solution for
this.  And I added the code in. 
So we will talk about it.  The 
debug class is a part of the 
Android API, and the debug class
and many of it's methods have 
been in the Android API since 
the first version, including 
these two.  
What the start tracing method 
function does is it asks the 
system to take a trace, save it 
with the file name, and it saves
it later so you can pull it up 
on the device later.  We will do
all of that work for you.
So if you are doing this 
manually, you probably would be 
very careful about the name that
you chose, if you are doing 
multiple chases, you would 
choose unique names so they 
didn't override each other.  In 
our case, the name is not going 
to show up in studio, so call it
CHARFBT  whatever you want.  I 
will put a recording around the 
function.  So in this case, 
there's an activity in the Santa
Tracker, the CD quiz which loads
files from the disk.  That's 
really good if you want to know 
how long it is going to take, if
it is suspicious, we want to 
take a look.  When you call the 
start method tracing function, 
it is going it to do a trace of 
your code.  This more expensive,
precise detailed one.  I would 
be careful, I wouldn't do this 
around a large amount of code, 
just to make sure it doesn't 
take longer than you would which
is down here.  We willexpect.
We will go to the city quiz,  
see what happens.  Before I do 
this, I want you to keep your 
eye over here in the session 
panel.  I will hit play.
It is going to run in, it is 
automatically recorded, you 
don't have to do anything, that 
is awesome.  Let me back utout. 
As you can see here, there is 
nothing new. 
This is what Shukang was showing
you before. 
And I think this is a good 
moment to call out the life 
cycle events I talked about in 
the beginning of the talk.  As 
you can see, we are doing a load
here.  We left the previous 
activity and we are into a new 
activity.  If you put your mouse
over an activity, as of Android 
Studio 3.3, we will include the 
fragments that are active during
that time.  That may be useful 
for you.  But we are in this 
space where we are loading 
before we entered the activity. 
And there's another really great
feature I want to show you here,
this filter button.  When I 
press the filter button, it 
brings up a search box, what I 
type into it.  And I happen to 
know that JSON has a function 
called read literal, let me type
that in.  And one thing you 
might notice here is that part 
of my call chart dimmed out.  
And all parts of here are not 
dimmed, they are the normal 
color.  I will zoom in and find 
where that is.  So you can see 
instances of the read literal 
function.  And basically, if my 
function is an exact match, it 
will bold.  If it is a function 
that calls either into that 
function or is called out of 
that function indirectly, the 
color will be left the same, and
otherwise it will dim.  And it 
is really useful to sort of get 
a good overview of how much time
you are spending in your code on
the function that you care about
versus what you don't have to 
pay attention to.  All of the 
CPU detail views support this.  
So the flame chart has the 
similar dimming, the top will 
strip out the functions that 
were dimmed.  If you are trying 
to inspect some sort of method 
and you are narrowing down on 
it, please give the filter 
option a try and see if you can 
sort of focus on what you are 
looking at. And the last thing 
to call attention to here is all
of the traces that you have 
done, if you mouse over them, 
there's a save button here, so 
you can export your traces.  If 
you do this, you can give it to 
a co-worker, attach it to a bug,
useful things like that.  If 
someone gives you a trace file 
or you are loading it, you can 
hit the plus button over here 
and load it from file.  Okay, 
cool.  
SPEAKER: So now I'm going to 
talk about the last type of CPU 
recording, the trace system 
cost.  And the trace system 
cost, this feature was 
introduced in Android Studio 
3.2.  It collects fine-grain 
system events related to app 
performance so you can 
investigate how your app 
interacts with the system 
resource.
So let's create the system
trace.
So I'm using the click to zoom 
button here, it is easy to see. 
And one thing I want to show you
here, the first thing I want to 
show you is in this thread state
view.  If you
click the range, I will zoom to 
that collection again.  I will 
click here.
So here, as you can see, you can
use your mouse, you can hover 
over and see this thread space. 
It is running, and it is runable
and running again.  We are 
collecting every CPU scheduling 
operation.  At these levels of 
details, it is easy for you to 
figure out where your thread 
becomes blocked.  And that could
be useful if you have some 
threading issues.
And another thing that I want to
show using the trace system 
calls is to investigate slow UI 
chunk.
So slow UI rendering, also 
called Jenk by some people.  As 
you may know, that's the UI, the
Android UI, it does the work in 
two phases: The first phase 
happens in the main thread.  It 
determines what is on the 
screen, including the
layout and the source.  It 
executes all of the UI elements,
such as all of the view classes 
in your app.
So after the main thread 
generates the what, they are 
passed to the native render 
thread.  The render thread will 
be figured out how to draw them.
And then the how is passed to 
the surface finger system 
process, and the hardware that 
is actually performing the 
joins.
So that is giving that 
background, so you can see under
the frame area, we have -- if 
you pronounce the main thread, 
the render is the render thread.
So let me zoom out a little bit.
So you can see here this -- this
the first phase, and 
corresponding the second phase 
will be here.
If you are targeting a smooth 
UI, the smooth animation at 60 
frames per second, which is 16 
milliseconds, roughly 16 
milliseconds per frame.  The two
phases combine together should 
be under 16 frames.  If it is 
longer than that, then the 
profiler will color that frame. 
So, you know, this is something 
--.
  So if you zoom out more, you 
can see a lot of frames, I think
that
everyone today is red, so 
exceeding the 16MS threshold.
One factor is that we are using 
the emulator.  Because the way 
the emulator interarcts with the
system, you are going to see 
more red frames from the actual 
device.  Before today's talk, I 
have collected another trace 
using a physical device.  I have
exported that trace as a file.  
And now I want to import that 
trace to show you.
So, as Dave said before, we can 
use this to import
the trace.  
And because when you trace the 
system course, you need to tell 
the profiler which process you 
want to look at.  So if you want
to look at the static tracker, 
as you know, the Linux from the 
system point of view, every 
process or every thread that 
your name is no longer than 15 
characters.  That is why this is
the Santa tracker.debug, for 
some reason, it thinks this is a
name.  If you select this one, 
you are going to import this 
trace.  And here, you can see 
from the actual device, most of 
the frames will be in gray, and 
that indicates they under 16MS 
threshold.  Some of the frames 
are in the red, because they are
over
that threshold.
And you may wonder how does the 
system or the profiler know how 
long the phases are taking.  
That is from the trace in point.
So Android platform engineers 
have added building tracing 
points into the sum of the 
critical tasks in Android 
system.
So, I mean, the example here is 
if you click the main thread 
here, these events are showing 
from the trace events here.
So if I zoom
in here, you will see this -- 
theseeral trace event called 
choreographer two frame.  That's
the first phase in the UI 
rendering, it happens in the 
main thread.  If you click the 
render thread, you can see there
that it is called a jaw frame.  
That is the second phase.  You 
can see that there are other 
tracing points in the system.  
They are all built in the 
system, they are available on 
any Android phones because they 
are building.  They are useful 
to get timing information for 
specific tasks, and actually you
can have your own trace point, 
too.  I'm going to demo that 
here.  So today, here, if we go 
to the top of the app, we have 
the cloud, we have the snow 
flakes.  I want to know how long
my code is spent drawing them.
So for the cloud, I go to the 
village view.
So there should be an on draw 
method.  So at the beginning of 
the method, I add the 
instrumentation, the trace
begin section, you need a 
stream.  And at the end of the 
method, we end the trace event. 
For the snow flock  flakes, it 
goes to the view, the on draw 
method.  I want to point out the
section name, you should pick a 
name that makes sense to you so 
you can recognize when you are 
doing the CPU recording.  Here, 
we are doing the snow flake, 
Android summit 2018.  That makes
sense for today's demo.
And so now, I will add the 
manual instrumentation as we 
view this app and reprofile it.
So while we are waiting for the 
build, you may be wondering, if 
I want to know how long those 
two methods are taking, use the 
trace Java methods.  That's what
we were talking about moments 
ago.
So trace Java method is very 
powerful, it is very easy to 
use, but it has significant 
overhead because the Java 
virtual machine is collecting 
data, every time when the 
execution enters the method and 
every time when the execution 
acts on the methods.  So if you 
have a lot of frequent, small 
methods, that overhead can 
quickly add up and become very 
expensive.  If you use the 
manual instrumentation, for 
example, like the trace -- using
the trace API, you will have the
full console of where and when 
to collect data.
So if you use that wisely, the 
overhead will be much smaller 
and, as a result, the data you 
cleck  collected will be more 
accurate.  So let's verify the 
instrumentation you have.  You 
go to the CPU profiler.  We 
collect the trace
system calls.
And so here, this is the view 
thread that is responsible for 
all UI elements.  And if
you keep zooming, you can see 
the village view, who is joining
the clouds, and this is a
snowflake -- what is responsible
for drawing the snow flakes.  
This is the trace system called 
CPU recording.  You may hear a 
very similar tool, sis trace.  
And my Google co-worker gave a 
lightning talk yesterday on the 
sis trace.  So it is an 
extremely powerful tool, but the
learning curve of the sis trace 
is a little bit steeper than the
Android Studio profiler.  So you
may choose the tools that best
suits your needs.  So I will 
hand over the demo back to 
David.  
SPEAKER: All right.
So let's leave the CPU profiler 
behind and jump over to the 
memory profiler.
First of all, I want to draw 
everyone's attention to this 
allocation tracking pull-down.  
Some quick history here, in 
Android Studio 3.0, when 
targeting devices with O or 
newer, we collected a call stack
for every single allocation that
your app made.  We did it 
because it would be convenient 
to use to have that history, 
however, some of our users 
reported that profilers were 
slowing their app down.  After 
we investigated, it turned out 
to be this feature.  So starting
in Android 3.3, we now give you 
the option to configure this.
And let me go ahead and look 
here.  So we have none, sampled,
and full.  None disables the 
feature, null enables it, and 
sample attempts to collect a 
subset of the allocations that 
would sort of gives you a 
general look for how your
ach's memory behavior is, 
without affecting the 
performance as much.
And now, that being said, 
without or not full affects your
app depends on the host machine 
you are running on, whether you 
are targeting the device, or the
code.  If your app code has a 
lot of small allocations on it, 
like Santa tracker does, it can 
be slow.  But I recommend 
playing with the features.  I 
will go and turn full on.
So notice once I turned on full,
the allocation indicator started
showing up.  I wanted some 
interesting things to happen so 
I will rotate the
device.
So I will rotate it back.  We 
will go and take a look here.  
So all I need to do at any point
is just to drag across, and I 
will be able to see all of the 
allocations and the 
de-allocations that happened 
during that range.  This can be 
a lightweight way to sort of get
a quick look at what your memory
is doing.  There is an 
allocation and a de-allocation 
count.  Sometimes you might find
memory leaks just doing this. 
And the other nice thing is, if 
you have this on and you are 
doing other stuff, you can go 
back in time and select a range 
and still see what is going on 
in memory.  I will go ahead here
and turn it back off again.
And you will notice that the 
allocation indicated that it 
stopped tracking by doing an 
empty circle.  If I did sample 
mode, it would put a half-filled
circle.  Now, looking at these 
objects, once I click on it, you
can see every single one that is
currently allocated as well as 
the -- where it was allocated.  
This is very useful to get your 
handle on a code base.  You can 
click on this and see where the 
different objects are coming 
from.  That being said, it might
not help you understand why a 
memory leak happened, what is 
holding on to my memory.  If I  
you want to get that 
information, you have to go over
here to this icon and click the 
heap done.  So what I'm going to
do here, just to let you know, I
could not find a leaking 
activity or fragment in the 
Santa tracker app.
So, full disclosure, I added 
one.  And what I did is I added 
a memory leak to this penguin 
swim game here.  So actually we 
will go ahead and go into the 
app, or the activity, and leave 
it.  And let me actually do 
another heap dump.
All right.
So we're about to get a bunch of
heat dump information here.  I 
know that the fragment inside 
this activity is called swim 
fragment.  Just like CPU, we 
have the filter
button here.  
And I can filter the different 
objects and sure enough, the 
swimming fragment is still 
alive.  I will click on it, 
select the active instance, and 
there's a lot going on here.  So
I will take a moment  moment to 
explain a little bit and take a 
moment to absorb it.  I want to 
explain what the depth idea is. 
So imagine all of your Java 
collector lives in this heap.  
There are special objects that 
live outside of the circle, 
called GC roots.  When you 
create a new thread, you create 
the GC root, or the static 
variables are an instance.  So 
what ends up happening, you in a
thread might create an instance 
of some class that creates inert
another instance of another 
class and you are creating this 
long chainf of objects and each 
item in that chain has further 
and further depth from the GC 
root.  You may have heard that 
cycles are not a problem to the 
garbage collector.  I can point 
at you, you at you, you at me.  
If we cannot all be reached, 
that is removed from the GC.  So
one of the things to note, if 
you take a heap dump, you might 
see a scary amount of things 
holding on -- in my instance, a 
lot of these are harmless.  A 
lot of them are potentially just
cycles.  And one way that you 
can kind of know, it is a good 
heuristicheuristic, the depth is
greater than your own depth.  So
imagine the garbage collector 
root points to you, you create a
child instance and one of the 
things you do, you give it your 
this pointer, it is pointing 
back at you, you are pointing at
me, you are my child and know 
who my parent is.  My depth is 
going to be added on to this 
pointer's depth.
So that's why some of these -- 
you are noticing it here, but it
is not an issue. And you are 
seeing columns where the depth 
is blank.  There is flow way to 
reach these at the moment from 
any garbage collection root T. 
Will be cleaned up, even though 
it is in the heap dump right 
now, you don't have to where he 
worry about it now.  Just ignore
it.  And just a concrete 
example, I will look here at the
code which is not the cause of 
our memory leak.  Our depth is 
one, and this item's depth is 
two.  So what this is saying, 
somewhere there is an instance 
of a score view class and that 
score view class has a variable 
called share/click/listener 
whose value is me.
And we will go ahead and take a 
look at swumming fragment.  And 
I can search for the score view.
And before when I say this, 
there has to be a score view 
that points at me, and it is 
inside the swimming fragment.  
If I jump to it, I create one 
and I pass in the this pointer. 
So that is explaining why 
there's a cycle there, I don't 
have to worry about it and it is
explaining why it is showing up 
in the heap dump.  And there's 
lot of this dollar sign zero 
symbols.  What is going on here,
you are going to see it a lot, 
you know what the this pointer 
is.  If you are an inner class, 
or a nested class and you need 
access to the outer class's 
fields, the compiler generates a
synthetic variable and instead 
of calling it this, because that
is taken, it is this dollar sign
zero.  That means that it is the
this, the one level out of my 
scope.  And anytime you create 
an inner class that has a 
reference to the outer class, or
you create a closure and an 
anonymous class instance, it is 
going to have that.  If you are 
using Lambdas in your code or 
anonymous classes, you will see 
a lot of zeros.  This has a 
depth of zero, with a GC root, 
that is suspicious.  We will 
look at the source.  And I might
look at this and say, where  I 
wrote this long running class.  
It is not static, it is final, 
and it will hold a reference to 
the parent class.  That's what 
is going on here, and this 
long-running task, I created it,
but I forgot to cancel it.  We 
will look here, sure enough, I 
left it out, for no if reason.  
I will uncomment it there and we
will re-launch.  So what we're 
going to do now is, I will make 
sure it is saved.  There we go.
What we're going to do now is 
reboot it just to make sure 
that, in fact, the swimming 
fragment was released.  One of 
the things to keep an eye out 
for when you are hunting for 
memory leaks is static variables
or singleton classes that are 
holding on to your class, or 
registering yourself with a 
listener and for getting to 
remove it, or any inner classes 
that, for some reason, may not 
end up stopping and they are 
running even though the activity
is trying to exit.  Are we 
profiling here? Yes.  We still 
going.
And then another thing I want to
mention is it is not always 
going to be this easy.  You are 
not going have this obvious 
culprit.  So, in that case, you 
are going to want to -- I would 
say just try to get to know the 
code, look for those things that
I talked about, we will go ahead
and enter.
And we will wait here while I 
talk.
And, you know, if you cleaned it
up, even if you didn't find it 
in the heap dump, it is going to
be the source of truth.  It is 
going to be the thing that
gar  guarantees your memory is 
reclaimed.  So I will go back 
into the memory profiler, click 
on the garbage collector.  You 
may have noticed these garbage 
collected events, automatically 
at the bottom.  So for example, 
there's a lot there.  That's 
where the garbage collector 
decided to collect on its own.  
And you can click the garbage 
collection button to manually 
cause it to get run.
And now we still see swimming 
fragment here.  We will see if 
the depth is actually -- as you 
can see, the swimming fragment 
is showing up I could be 
nervous, but the depth column is
blank.  And everything else is 
waiting to be picked up by the 
GC.  You can press the garbage 
collector a few more times.  And
one trick I like to do is rotate
the phone, rotate it again.  And
every time you rotate an Android
phone, a lot of fun things 
happen, it tells the GC a lot of
things are going on.  We will 
see the heap dump and assume 
that it is truly well and 
collected.  I wish it came with 
a drum role, and  roll, and it 
is gone.  If you are looking at 
the heap dump, you are not 
necessarily hunting memory leak,
you might not know a major class
to look out for.  There's the 
concept of shallow size and 
retain size.  Shallow size is 
the size of a single instance of
some class that it have been 
allocated.  And retain size is 
all of the things that it is 
holding on to.  So what you may 
want to do is hunt, sort the 
shallow and retain size and 
investigate to see if there is a
suspicious memory thing there, 
you can clean up your design and
remove memory there earlier.
So that ends the demo, and back 
to Shukang.  Let's go to the 
slides.  
SPEAKER: So to recap, in today's
demo, we showed you how to use 
CPU and memory profiler to get a
better understanding of the code
base for the Santa Tracker app. 
To be honest, Dave and I don't 
know much about it before we are
preparing for this talk.  And we
also have shown that we don't 
only use profilers to diagonal 
performance issues, we use them 
to help us understand the 
performance of this app.
And as we said before, there are
also network and energy 
profilers in Android Studio, but
unfortunately we don't have time
to cover them today.  So these 
refer to our online documents 
and talks to learn more.
So we hope we have learned some 
tricks and tips from our demo 
today.  We hope that you  they 
are useful when you approach 
your own code base with Android 
Studio builders at your side.  
Thank you for attending our 
talk.
  [ Applause ]
  SPEAKER: The next session will
begin in 10 
minutes.
Coming up next: Testing Android 
Apps at Scale with Nitrogen by 
Stephan Linzner, Vishal Sethia.
#
Stephan Linzner, and back with 
me
is Vishal Sethia.
And we develop products for 
everyone, and that means we have
to test for everyone.  So let me
tell you a little bit about how 
the development test happens at 
Google.  Before I start, I want 
to talk about the scale that we 
do on Android development at 
Google.  We have 100-plus 
Android apps, including the user
apps such as Google photos, 
maps, YouTube, Gmail, search.  
We have 200 billion lines of 
code, and run 20,000 Android 
builds every day and have a 
staggering 27 million test 
innovationss -- invocations per 
day.  How do we create these 
high quality app and maintain 
that quality over such a
long time?
An a typical developer workflow 
looks like this.  We have a 
strong code review culture, code
reviews are very, very thorough 
and, before you can actually 
submit your change or pull 
request, you have to get at 
lease reviewed by one of your 
peers.  And another important 
thing is all development happens
at head.  We have a large repo 
that allows us to search for and
use code and create large-scale 
changes.  And we also have a 
very strong testing culture.  At
Google, if you have a change, 
you have to have tests.
And even more importantly, all 
of those tests have to pass 
before you submit your change.
To run tests, you use a 
large-scale distributed CI 
system, which not only runs your
tests, but all of the testss 
from code that depends on your 
change. Another thing that is 
very unique about Google is we 
have a strong engineering 
productivity culture.  That 
means we have dedicated teams 
that only work on infrastructure
tools and APIs to make 
developers 
productive.
We are part of such a team, and 
we have been working on testing 
Android apps, or Android app 
testing, at Google.
So I want to take you a walk 
down memory lane what we have 
done at Google to scale Android 
testing.
In 2011, a lot of teams at 
Google actually started to build
for Android because Android was 
becoming more and more popular. 
They were using a standard tool 
chain, ant and eclase.  And we 
support said ed the internal 
build system.  A problem that 
was obvious early on is the need
for scalable testing.  We 
started off very simple by 
building a small host-like test 
runner that would run on the 
host.  And, in fact, it was a J 
unit three test suite, list the 
APK, list the tests, and give it
on the instrument test runner on
the device to execute the test. 
And once we have that, we built 
it into a continuous integration
system.  One of the key 
decisions we made early on was 
to use emulators, we called them
virtual devices, to run tests 
alts at scale because it makes 
more sense because you can scale
a data center, but not a USB hub
so easily.  So we wrote this 
Python script, 200 lines of 
code, you probably have been 
there, that boots the emulator 
for us that allows us to run the
tests and shout it down 
afterwards.  While we were 
working on infrastructure, our 
engineers started to write 
tests, and they wrote a lot of 
them.  A key problem here is 
around functional UI testing. 
In the early days, you had 
low-level framework APIs, you 
are monitors to track 
activities, or wait until idle 
sink  sync.  And even though 
these APIs were easy to use, 
developers struggle a lot 
writing reliable UI tests.  We 
thought that maybe we can find 
something better, and we did
in the community Robotium, and 
we used them until 2012.  It had
issues bes with the API surface,
and it did not solve
synchronization.  We used 
Espresso, we wanted something 
easy for developers and handled 
the complex instrumentation 
testing from the developer.  We 
had a decent set-up for 
instrumentation tests, but we 
have to solve the unit testing 
problem.  At the time, all of 
the unit tests, you ran them on 
the device.  But that is 
expensive and they tend to be 
slower than running on the JVM. 
So we reached for a solution, 
which at the time was 
RobolectricRobolectric.  It 
allowed for fast local 
development and is still one of 
the most popular frameworks for 
unit testing in Google.  In 
2014, we build a lot of 
experience in testing 
APIs, but we were seeing that 
the community was struggling 
from the same problems that we 
did.  
So we bundled all of the 
libraries together in the 
Android testing support library 
, the default library for 
developers writing 
instrumentation tests.
Fast forward to today, we 
launched AndroidX test 1.0.  It 
is not only the first stable 
release, it is the first time 
where we ship unified APIs that 
allow you to write tests once 
and run them anywhere.
And, by the way, we just 
achieved a major milestone at 
Google.  We run 10 billion unit 
and instrumentation tests every 
year on our infrastructure.
And so, looking back at those 
seven years, what will go 
differently? There's a couple of
things I want to mention here.  
So we can probably design for 
any build system.  We made some 
key decisions early on that 
tightly coupled us to Google's 
internal build system.  But this
was a problem, even at Google, 
not everyone is using Google's 
internal build system, and we 
were not able to host that 
infrastructure with them, and 
not with the community and we 
could not open source it.
Similarly, we did not build some
of the tools -- some of the 
tools that we built were not 
cross-platform, they worked on 
Linux, but not Mac and Windows.
And another thing we would do 
differently, and 
retrospectively, it is not a 
good thing that we started out 
small and scaled up our testing.
While the apps grew and the 
ecosystem grew, there were more 
and more requirements.  We 
usually built them into the 
infrastructure, but we did not 
have a mechanism for teams to 
customize the infrastructure.  
We suffered from hide code 
complexity t was hard to 
maintain and some features would
thought could not be removed and
they were not used anymore.  The
other thing to mention is 
configuration, the hosting 
infrastructure was getting 
configuration from many sources,
flags, system environment 
variables and config files.  
This made it hard to track down 
bugs in the infrastructure 
itself.  So about a year ago, 
our team sat down with app teams
in Google and we wanted to learn
about the past and the future 
and especially how the Android 
testing landscape had changed. 
And so what we came up with to 
solve  solve some of the 
problems out of the discussion 
was project nitrogen.  It was a 
new unified testing platform 
that we first talked about at UI
UAT I/O this year that will ship
in 2019.  Project Nitrogen is 
used by a small number of apps 
in Google and we are slowly 
scaling it up to some of the 
bugest in the world.  The reason
we are doing this is because we 
want to battle test it first 
before we ship it to you.  But 
the point is we want to give all
of the infrastructure that we 
use to run 10 billion tests to 
you.  So nitrogen solves many 
problems, but two of the key 
issues that we are trying to 
solve with nitrogen is, first, 
we want to create a unified 
entry point into Android 
development.  And secondly, we 
want to be able to write tests 
with a unified API and move them
between layers.  If you think 
about Android testing today, it 
looks like this you. Have.  You 
have tools on the left, such as 
Android studioStudio, Gradle, a 
CI system, and Bazel.  On the 
right, you have run times you 
want to run on.  We call run 
time devices on Nitrogen.  You 
want to run tests on a simulated
device, or a virtual or physical
device, or on a remote device 
that runs on a device lab, such 
as Firebase.  And in order to do
so, we have many different entry
points, it looks like this.  You
have a configuration for every 
tool, you have different roles 
and tasks and it is a nightmare 
to maintain.  What we see in 
Google, it is so hard to move 
from one to another, they would 
go from one test to another.  So
what we want to do with nitrogen
is to have a unified entry 
point.  And nitrogen itself is a
stand-alone binary, or tool, and
infrastructure developers can 
use to customize their 
infrastructure.  And obviously, 
there are all of these other 
developers that don't work on 
infrastructure and work in 
actual lab code.  We want to 
provide integrations into all of
the tools on the left hand side 
to make it easy to run tests.
And, at that point, if you have 
a single entry point and a 
unified test, it fits very well 
within your developer workflow, 
because you can do local, fast 
iterative development on a 
simulated device.  And before 
you submit the change, you can 
run on an emulator matrix.  And 
lastly, in post-submit, you can 
run on a remote device, a 
physical device, in Firebase 
test lab.  And that is what we 
are trying to do with nitrogen.
Nitrogen allows you to run tests
at scale.
It is highly configurable, it 
was build with customization and
extensibilityextensibility in 
mind.  You can execute unit 
tests, it improves reporting 
and, therefore, debugging.  And 
one of the most exciting things 
is it ships with its virtual 
device management solution that 
manages devices for you.
And that is actually something I
think a
lot of the community has been 
asking us for quite a while.  So
Nitrogen is cross-platform, and 
we build it from the ground up 
with all the experience that we 
have, seven years in host side 
in that infrastructure.  It 
supports Mac, , Windows, and 
Linux, and it is written in 
Kotlin, and we write it in a way
that it will be viable for the 
next seven years.
And
Nitrogen is a stand alone tool. 
It can be integrated into any 
system.  We are working on 
integrations for Gradle and 
basilbasil Bazel.  We are 
working on test execution and 
support will be there from the 
start.  On the device side, we 
are planning to have support for
simulated, virtual, and physical
devices, and device labs, such 
as Firebase.  And you can even 
add your custom devices if you 
have custom hardware.
So let's switch gears a little 
bit and talk a little bit about 
the high level architecture of 
nitrogen.  So nitrogen is 
basically split into two parts: 
We have a host side 
infrastructure that is all the 
code that runs on the host, ask 
we've done something new.  We 
have an on-device infrastructure
which means that we have moved 
some of our infrastructure on to
the device, which is a much 
saner environment to reason 
about, and the device is the 
main abstraction that we use in 
nitrogen for different run 
times.  So the host site runner 
is mostly responsible for 
finding the device for you, 
setting up the device for test 
execution, and then requesting a
test run.  It can be easily 
configured with a proto buffer 
configuration, and it allows you
to customize things like the 
test executor and the whole test
harness.
To decouple the host from the 
device, we have a new 
orchestrater service.  You can 
think of it as the brain
is responsible for discovery, 
filtering, and sorting, and 
execution.
And an orchestrater service is 
just a GRPC service that can be 
implemented by any device.  And 
we, in fact, use GRPC to 
communicate between the host and
the device, which does not give 
us performance and speed.  It 
gives us a lot of stability and 
it allows us to stream test 
results back to the host in 
realtime.  We also have a lot of
extension points.  So we will 
have host plugins that allow you
to run code on a host, and we 
will also have device plugins 
that allow you to run code on 
the device.  So let's dive into 
each of these sections.
As I mentioned before, we use a 
single proto configuration with 
a declarative set of protos.  
This allows you to find devices,
the text features, so you can 
find APKs that you want to 
install, and it can declare your
host and device plugins.  We 
initially will have support for 
single device executers, 
parallel device executers, to 
run on multiple devices in 
parallel.  And we will also have
a new multi-device executor, 
which will allow you to do 
things like orchestrating a test
run between a device and a 
device, or a device and a watch,
which is something that we 
increasingly see as a 
requirement.  The good news is: 
If you are just an app 
developer, you don't have to 
deal with any of this 
configuration, because it is 
built in in the tool 
integration.  But if you are an 
infrastructure developer, this 
is where it is really 
interesting for you.  Because 
you can customize every single 
bit of Nitrogen.
Let's talk a little bit about 
plugins.
So host plugins are plugins that
can execute code on the host.  
Plugins that we've already built
are the Android plugin, theythey
encapsulate all the code that 
allows us to run Android tests 
on the device.  We have a data 
plugin that allows us to stage 
data on the device, or a 
descriptor plugin that allows us
to execute fixer scripts on the 
device.  And you can have your 
custom plugins.  Custom plugins 
can have their own 
configuration, and with host 
plugins, you can run before the 
test suite starts, and after the
test suite is finished.  The 
reason we do it this way is we 
want to avoid the chattyness 
between the host and the device.
If you look at the after all 
method, you will also get access
to the host test suite result, 
which is great if you want to do
any post-processing of your test
results.  And you can submit an 
added reQuestback to us if you 
want to attach new artifacts to 
the test suite result.  Device 
plugins, on the other hand, like
the name says, are running on an
actual device, which is a much 
more sane environment to reason 
about.  Most of the host side 
code that we use to configure 
the device is moved to the 
device with a device plugin.  So
plugins we have built are a 
lockhead plugin, a screenshot 
plugin that talks screenshots in
case the tests fail, or a 
permission plugin was pretty 
awesome.  You can grant and 
revoke run time permissions, 
which was not able before, and 
you can obviously also have your
custom plugins.
So the difference from the 
device plugin to host plugin is 
that it runs on a device.
But this allows us to do things 
like that.  We can give you a 
call-back before the single test
method
is executed, and after it is 
finished.  This is great, we can
avoid the chattyness between the
host and the device, and it 
gives you a lot of control.  And
if you think about it, I don't 
know how you set up your test 
features now, but you generally 
have something like add before 
class, or add after class.  If 
you want something reusable, you
will probably reach for a J unit
rule, or there is some things 
that you can't do with these 
APIs, and then you have to have 
your custom runner.  And I think
the great thing about this is, 
we give you a whole new way of 
writing plugins that run on the 
device and allow you to execute 
code on it.  We will move to 
execution.  As I was saying, we 
move the execution to the actual
device.
And we created a whole new 
orchestrater service and 
protocol.
What this does, it standardizes 
the communication between the 
host and the device and it can 
be implemented by any device, 
which means if you have a custom
device, you can implement the 
same protocol, and you can still
integrate with the host site 
easily.  On Android, it is 
implemented by the Android test 
orc  orchestrater.  Once you 
request the test run on the 
host, it will go and discover 
all the tests, apply any filters
and sorting that you want, and 
then it will do either isolated 
or batched test execution.
It will also call all of your 
device plugins, and it will 
stream results back in realtime 
to the host.
So the last thing that I want to
talk about is reporting.  So 
with nitrogen, we will give you 
unified and consistent 
reporting.  As I'm sure many of 
you have seen this command at 
the top, what it does is it runs
an instrumentation test from the
command line.  If you use the 
dash R option, you will get an 
output like this.  As you can 
see, it is not very human 
readablereadable, I would say, 
and it is also quite chatty, 
because this is just showing a 
sungal test.  
And this is showing a passing 
test.  If it fails, the only 
thing that it gives you an 
addition is a stacktrace.  So 
there is not really a lot of 
information or actionable data 
here to why the test failed.
And with nitrogen, we want to 
move to something like this, a 
structure data format, which 
gives you access to the 
properties of the test case, the
status of the test, and a list 
of art  artifacts collected 
during the test run.  Things 
like screenshots,
video, lock, and any custom 
artifacts that you run in your 
post-processing.
Again, this will also be 
integrated in Android Studio, 
and we will surface this in the 
Android Studio UI if you run 
tests.
The last thing before I wrap up 
what, I wanted to mention we 
have support for custom reports.
You can do J unit XML, or the 
custom report that integrates 
better with your own 
infrastructure. And with that, I
want to hand over to 
Vishal who is going to talk 
about device management.
[ Applause ]. 
SPEAKER: Running any kind of 
Android UI test generally 
happens on devices.  There are 
two device tests where you can 
run the test, the physical or 
the watchful device.  
Regardsless, they each have 
their own sets of pros and cons.
Let's do a quick show of hands. 
How many people around here have
set up something like this, 
testing on physical devices? It 
looks like quite a few.
So a follow-up question.  How 
easy was it to manage them?
Hard.
And another follow-up question. 
Did you end up using a file 
testing machine next to it? I 
hope not.  I have a funny story 
to share that happened a few 
years at Google when one of the 
teams decided they wanted to 
test their stuff on physical 
devices.  They procured a bunch 
of devices, it  glued them to 
the wall, and integrated with 
the CI infrastructure.  
Everything was running well 
until one fine day when the 
engineers came back on a Monday 
morning and things timed out.  
If you were to guess what went 
wrong, what would your guess be?
Okay, so it turned out to be an 
air conditioned problem.
So what happened is the air 
conditioners in the building in 
San Francisco went bad, and 
because the air conditioners 
went back, the facilities 
decided they wanted to switch 
off the air conditioners so they
can fix it, but tests were 
running on those devices and the
heat produced in the devices 
caused the glue to peel off from
the wall and all the devices 
fell to the ground.  Managing 
physical devices are hard.  I 
want to give a huge shout out to
the fair base test lab team that
makes testing on Firebase so 
much easier for you folks. How 
do we solve this at Google? The 
virtual device infrastructure, 
the testing environment we use 
is very stable.  The number on 
the right is the stability ratio
of our test environment, it is 
99.9999 percent.
The continuous integration, the 
virtual device infrastructure 
that we use has the ability to 
run locally, or in a CI 
environment.  And it supports 
over 500 different device 
configurations.  Let's see what 
is the current state at Google. 
It used be over 100 first party 
apps, such as Google photos, 
search, YouTube, and so on.  
Just in 2018, it had a 
staggering 2.4 billion 
invocations, that number is 
growing year over year.  There 
are 28,000 targets using this 
infrastructure.  Having a great 
infrastructure is a must for 
volatile apps.  How does this 
fit in with Nitrogen? If you 
remember from slides presented 
earlier, NIFEP  nitrogen has the
concept of device protide 
viders, the virtual device 
provider, which launches the 
device, returns the concept back
to nitrogen and executes the 
test.  Ask  once the test is 
done t , it starts the device.  
You get a stable environment 
launched by nitrogen, runs the 
test, and shuts it down.  And 
while designing this 
infrastructure, there are four 
things that we kept in mind.  
The virtual device 
infrastructure needs to be 
simple to use, it needs to be 
extremely stable, and you should
be reproducible, regardless of 
the environment running in, 
locally or in the CI 
infrastructure, and it needs to 
be fast.  We will dig into it 
deeper as to how we achieved 
each of these four goals.
So the virtual device 
infrastructure has a very simple
protocol configuration.  What 
does that mean? It is a config 
file where you can add the 
characteristics of the device.  
What is the horizontal scheme 
resolution, the vertical screen 
resolution, the memory of the 
device? So for each of the 
device types, like nexus and 
pixel, the device management 
solution has pre-baked in all of
these device configurations, you
don't have to figure out the 
different resolutions for each 
of these devices.  It supports 
over 500 different device config
RAISHZ figerations. Because it 
is a file, it is a matter of 
adding or removing changes to 
the configuration file.  It
has several different -- and 
launching it is as simple as 
calling the device binary and 
specifying the name of the 
device.  If you want to launch a
pixel two, device equals pucks  
pixel two and the API level.  
You don't have to worry about 
specifying configurations and 
things like that.  That makes it
very simple.
  And stability, this is one of 
the biggest problems most of the
Android app developers face, you
are running the test and an ANR 
pops up and it might not be the 
app you are testing.  We had the
same problem internally as well.
How did we solve this?
Well, sorry, Android has a nifty
service to suppress ANRs when it
sees them.  This is the same 
service that Android monkey 
uses.  This increases the 
stability of the tests.  One 
thing consider  I forgot to say,
when we started with this 
infrastructure, the stability 
was 95 percent.  That is no good
when you are running things at 
scale.  The first thing we saw 
is ANRs, once we fixed it, the 
stability increased.  But not to
the level we wanted.
  The next thing that we saw is 
we booted up the device, but the
screen is not unlocked.  And if 
the screen is not unlocked, all 
the key elements that you inject
do not reach the app.  If they 
don't reach the app, they are 
not getting tested, and your 
tests are starting to fail.  And
when the device boots up, the 
screen is not locked.
So the careen is screen is not 
unlocked.  And Google added an 
API where you can dismiss the 
key guard to unlock the screen. 
So every time you call the 
device, we will call to unlock 
the screen.  A few years ago, 
Android changed the file system 
from via FFS, another file 
system, to EXT4. 
EXT4 was prone to discorruption 
during shut down.  So if we did 
not correctly shout it down, the
subsequent -- (indiscernible) --
of the device would fail, 
leading to test flakyness.  How 
did we solve this problem? We 
called in SSH to disk image that
was unmounted, and this 
guaranteed, when the disk was 
unmounted, it had no diserrors, 
if there are no disk errors, the
subsequent boot would come up 
fine, this increased the 
stability of the test 
environment.  But that is no 
good.  When you are running on 
2.4 billion 
innovations, a one-person 
failure is a huge number.  
There's a bunch of optimizations
we did for the stability.  I 
will not talk about all of them,
but there's one final thing to 
talk about.
Launch the device, it will set 
up a boot saying that the device
is booted up.  But for whatever 
reason, the launcher dudsdoes 
not kick in.  How did we solve 
that problem? We had to send out
an intent to the launcher, if it
was lost, if it wasn't launched 
then we started the launcher and
we would return the control back
to nitrogen that would run the 
tests.  Doing a bunch of 
optimizations like this helped 
us to get to 99.9999 percent 
stability.  The next big pillar 
we had in mind is reap 
reproducibility.  In the CI 
environment, if the test failed,
there is no way to debug it 
locally.  So the virtual device 
environment we built, we have to
make sure the environment is 
reproducible regardless of what 
they are running.  So the 
management solution helps you to
run things locally or on the 
cloud.  And one of the big 
things about this environment is
the device starts in a clean, 
pristine state.  So there is no 
state carried forward with 
different invocations, making 
sure the state is stable and not
failed because of the device 
itself.  Android Shell.  There 
are several teams within Google 
that indicate when you are 
writing native code.  To test 
the native code, we wanted to 
boot up on devices.  And booting
up on devices were slow.  On 
nougat, it takes 10 minutes.  
And this is slowing it down 
tremendously.  This made us go 
back to the drawing board to see
what can we do to decrease the 
time it takes to boot up
those devices.
So we ended up going and 
creating a miniboot mode in 
virtual device.  What does 
miniboot mode mean? You don't 
need the Android stack to be up 
and running, you need the Linux 
kernel, if it is up and running,
you can test the native APIs.  
We added it to the device 
launcher, comes up in less than 
30 seconds, allowing them to 
test the native code quickly.
 Because we were running things 
at scale, we were looking at 
where we were spenting our time.
25 percent of the time was spent
booting up the emulator, 
installing the app, and 20 
percent of the time was spent in
running the test itself.
Android made a change between 
lollipop and
nuugat, they wanted an error at 
time compilation.  
And because the stacks were so 
huge, we ended up -- you have 
the same device and ach on the 
test that is being tested and 
the same Dex2O at every session.
What if we moved this as a 
single action on the Bazel bell 
graph and reused the old file 
that was generated for the test 
runs? This reduced the install 
time from 3 minutes to under a 
minute.
When the emulator team presented
about snapshots, you can boot it
up, and shut it down, when you 
restart it, it restarts back 
from the same state.  We 
integrated the snap shot feature
into virtual device launcher, 
you boot it up, take a snap 
shot, shut it down and reuse it 
when the test runs. This reduced
test run times by 30 percent.  
When you run tests at 2.4
invocations, it yields a huge 
number of -- you save huge 
amounts of CPU resources.
One of the other features
we wanted to work next year is 
snapshots.
And this, we come to the end of 
the talk, with nitrogen, you can
run the test at scale in a 
completely stable environment 
with the different pillars. This
is the stabilization platform to
help you test.  In this talk, we
did a lot of technical stuff 
like activity controller, you 
don't have to worry about all of
those things, they are 
incorporated in Nitrogen and the
virtual device management 
solution, you just have to use 
this.
So we are hoping to release 
nitrogen alpha in Q1 of next 
year, and the virtual device 
management solution is going to 
be released the same time as 
well.  Firebase test lab is 
actually integrating with 
nitrogen as well to run your 
tests.  And one of the things 
that Steven pointed out earlier 
about integration with Android 
Studio, imagine you are sitting 
in Android Studio, you launch 
the device from there  the tests
and it give gives you the 
results.
That's it.  Thank you very 
much.
[ Applause ]. 
SPEAKER: Well, hey, everyone.
I wanted to thank you all for 
coming and attending the Android
Dev Summit.  It is really, 
really amazing to do this again.
We want to know what you think, 
this is really important.  All 
of you will get a survey in your
inbox.  Please fill it out, 
because it is -- we so much want
to make this event amazing, 
should we ever do it again, we 
really want to know what worked 
and didn't work, what you liked,
didn't like.  And the second 
thing is, we are going to -- we 
have these QR codes that we can 
put up one more time hopefully. 
And these are how you rate the 
sessions.  We want to know what 
sessions you loved, what 
sessions you liked, what 
sessions you sat through because
they were there in the same room
and you were kind of comfort  
comfortable in your seat.  
Please fill out the surveys, I 
know it is a lot of work, but we
appreciate  appreciate that.  If
you miss anything, all of the 
talks are up right
now on the Android developer's 
YouTube channel.  From yesterday
and most of them from today are 
already going up.  And by the 
end of tonight, all of them are 
on the channel.  You can go home
tonight, if you have not enough 
Android Dev Summit by now, you 
can have more from the comfort 
of your very own history museum.
And finally, we have a little 
bit of a final reel here of some
of what was going on here that 
we should share to you as you 
think about wandering out here 
and going back to the real 
world.  So thank you so much for
coming again.
[ Applause ]
