Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér.
In this series, we have seen many times how
good neural network-based solutions are at
image classification.
This means that the network looks at an image
and successfully identifies its contents.
However, neural network-based solutions are
also capable of empowering art projects by
generating new, interesting images.
This beautifully written paper explores how
a slight tweak to a problem definition can
drastically change the output of such a neural
network.
It shows how many of these research works
can be seen as the manifestation of the same
overarching idea.
For instance, we can try to visualize what
groups of neurons within these networks are
looking for, and we get something like this.
The reason for this is that important visual
features, like the eyes can appear at any
part of the image and different groups of
neurons look for it elsewhere.
With a small modification, we can put these
individual visualizations within a shared
space and create a much more consistent and
readable output.
In a different experiment, it is shown how
a similar idea can be used with Compositional
Pattern Producing Networks, or CPPNs in short.
These networks are able to take spatial positions
as an input and produce colors on the output,
thereby creating interesting images of arbitrary
resolution.
Depending on the structure of this network,
it can create beautiful images that are reminiscent
of light-paintings.
And here you can see how the output of these
networks change during the training process.
They can also be used for image morphing as
well.
A similar idea can be used to create images
that are beyond the classical 2D RGB images,
and create semi-transparent images instead.
And there is much, much more in the paper,
for instance, there is an interactive demo
that shows how we can seamlessly put this
texture on a 3D object.
It is also possible to perform neural style
transfer on a 3D model.
This means that we have an image for style,
and a target 3D model, and, you can see the
results over here.
This paper is a gold mine of knowledge, and
contains a lot of insights on how neural networks
can further empower artists working in the
industry.
If you read only one paper today, it should
definitely be this one, and this is not just
about reading, you can also play with these
visualizations, and as the source code is
also available for all of these, you can also
build something amazing on top of them.
Let the experiments begin!
So, this was a paper from the amazing Distill
journal, and just so you know, they may be
branching out to different areas of expertise,
which is amazing news.
However, they are looking for a few helping
hands to accomplish that, so make sure to
click the link to this editorial update in
the video description to see how you can contribute.
I would personally love to see more of these
interactive articles.
Thanks for watching and for your generous
support, and I'll see you next time!
