Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér.
When it comes to image classification tasks,
in which the input is a photograph and the
output is decision as to what is depicted
in this photo, neural network-based learning
solutions became more accurate than any other
computer program we, humans could possibly
write by hand.
Because of that, the question naturally arises:
what do these neural networks really do inside
to make this happen?
This article explores new ways to visualize
the inner workings of these networks, and
since it was published in the Distill journal,
you can expect beautiful and interactive visualizations
that you can also play with if you have a
look in the video description.
It is so good, I really hope that more modern
journals like this appear in the near future.
But back to our topic - wait a second, we
already had several videos on neural network
visualization before, so what is new here?
Well, let’s see!
First, we have looked at visualizations for
individual neurons.
This can be done by starting from a noisy
image and add slight modifications to it in
a way that makes a chosen neuron extremely
excited.
This results in these beautiful colored patterns.
I absolutely love, love, love these patterns,
however, this misses all the potential interactions
between the neurons, of which there are quite
many.
With this, we have arrived to pairwise neuron
activations, which sheds more light on how
these neurons work together.
Another one of those beautiful patterns.
This is, of course, somewhat more informative:
intuitively, if visualizing individual neurons
was equivalent to looking at a sad little
line, the pairwise interactions would be observing
2D slices in a space.
However, we are still not seeing too much
from this space of activations, and the even
bigger issue is that this space is not our
ordinary 3D space, but a high-dimensional
one.
Visualizing spatial activations gives us more
information about these interactions between
not two, but more neurons, which brings us
closer to a full-blown visualization, however,
this new Activation Atlas technique is able
to provide us with even more extra knowledge.
How?
Well, you see here with the dots that it provides
us a denser sampling of the most likely activations,
and, this leads to a more complete bigger-picture
view of the inner workings of the neural network.
This is what it looks like if we run it on
one image.
It also provides us with way more extra value,
because so far, we have only seen how the
neural network reacts to one image, but this
method can be extended to see its reaction
to not one, but one million images!
You can see an example of that here.
What’s more, it can also unveil weaknesses
in the neural network.
For instance, have a look at this amazing
example where the visualization uncovers that
we can make this neural network misclassify
a grey whale for a great white shark, and
all we need to do is just brazenly put a baseball
in this image.
It is not a beautiful montage, is it?
Well, that’s not a drawback, that’s exactly
the point!
No finesse is required, and the network is
still fooled by this poorly-edited adversarial
image.
We can also trace paths in this atlas which
reveal how the neural network decides whether
one or multiple people are in an image, or
how to tell a watery type terrain from a rocky
cliff.
Again, we have only scratched the surface
here, and you can play with these visualizations
yourself, so make sure to have a closer look
at the paper through the link in the video
description.
You won’t regret it.
Let me know in the comments section how it
went!
Thanks for watching and for your generous
support, and I'll see you next time!
