Hi. We're gonna do a very quick
introduction to two concepts: neural
networks, and back propagation. And here
we'll have a toy example of how a neural
network can learn, and to classify
features that it sees. By the way, if you
have taken machine learning before, this
is an optional video. This is just training
like a simple perceptron with back
propagation. So the first thing we would
need to do would be to define our
features. We need some arbitrary set of
features. Maybe it's the pixels of a picture, maybe
it's the wordVec for a word, maybe it's
some features that you defined. For
example, let's say we have two objects in
the world - in the world, kittens and
puppies. We're gonna call them just cat
and dog in this set. And we define two
toy features to identify them. Do they
have whiskers? Yes or no. And are they a
good boy? Yes or no.
And of course cats can be good boys but
for this example, let's just use that
English collocation for the puppies. When
we define our feature system, we need to
figure out the values of the features
for each of our inputs. For example, the
kitten has yes whiskers, so it has
whiskers equals one. And it is not a good
boy (asterisk of course kittens are good
boys) but for this example is a good boy
equals zero. So the feature values for
the kitten would be one, zero, it does
have whiskers, and it is zero is a good
boy. How about the puppy? It does not have
whiskers, so it has a zero for that
feature, and then it has a 1 for the
feature is a good boy. So for the kitten,
the features would be 1 0, and for the
puppies, the features would be 0 1.
We have our input but then we expect the
system to do something so that it
transforms one zero into, for example, a
one for cat, or we expect it to take
the features zero one and transform
those into a zero for dog. Let's
arbitrarily call cat one and dog zero
for this example. So again we want some
sort of algorithm that tra - that takes
one zero as the input and transforms it
into a one, or that takes zero one as the
input and transforms it into a 0 and we
have encoded cat as the output, 1 and dog
as the output 0. So we want something
that transforms this input into that
output. We're going to use something
called a single layer perceptron. A
perceptron. The perceptron has the
structure that you see in the upper left.
It has an input layer, as you can see, the
input layer has two neurons. One of them
takes the value of has whiskers for each
of the feature vectors, and the second
neuron takes the value of is a good boy
for the feature vectors. So it has an
input layer where one neuron takes the
value of has whiskers, and the other
one takes the value of is a good boy.
This is the first thing that this
structure does. The second thing that it
does is that it has weights that connect
the neurons to the input layer. The - the
neurons are the input layer to some next
layer. Let's call this one an output
layer for now. How do these weights work?
They're gonna have some randomly
assigned weight. When we start the perceptron, 
we assign these values at random.
So 0.93 for the weight from has whiskers
to the output, and 0.06 for the for the
neuron of is a good boy to the output.
And so what value travels from input to
output? In the case of cat, we would need
the input for has whiskers, which for cat
is a 1 multiplied by the weight which is
0.93, and that's what transferred here
for the value of this neuron. In the case
of the second one, we need the value of
is a good boy multiplied by the weight.
And the summation of these two things
it's what comes here. So as you can see
what gets transferred from the input
layer to the next step is the value of
cat for has whiskers 1, multiplied by the
weight for has whiskers to the next
layer 0.93, plus the value for the
feature is a good boy in the input layer,
multiply it by the weight for that
neuron from the input layer to the next
layer,
0.06, likewise for dog.
So because cat has a value of 1 for
has whiskers, this is 1 multiplied by
the weight from has whiskers to the
next layer 0.93, plus the value of is a
good boy for a cat 0, multiplied by
the weight of that neuron into the next
layer, 0.06. So this is 1 multiplied by
this, plus 0 multiplied by this 0.93, so
0.93 is the value that goes from 1 0 and
then the weights and arrives here.
For dog, we have that has whiskers is
zero, and that is a good boy is 1. so 0 x
0.93 plus 1 x 0. 06 equals
0. 06. So we have two inputs. The
value for has whiskers and is a good boy, and
then this produces a value that is
transferred on to the next stage of this
network, being multiplied by the weights.
In this layer, we're going to take this
value which can have decimals, and we're
going to perform something called an
activation function which can't have
decimals. But in this toy example we're
going to use a very simple activation
function. If the value we get is less
than 0.5 so that's in half of 1
we're just going to round it to zero. If
the value we get is more than 0.5 we're
going to round it to 1. So it's just
rounding to the nearest integer. If we
get 0.9, for example, we're going to round
it up to 1. If we get 0.06, we're gonna
round it up to 0. Indeed this is what we
did. Because cat gave us 0.93,
here we pass it through the activation
function, and this is gonna give us an
output of 1, and because dog gave us a
value of 0.06 here, when we pass it
through the activation function, is gonna
give us a value of 0. So what happened?
Did this thing work? It did.
When we gave the structure the values
for cat 1 and 0, 1 multiplied by zero
point nine three, zero by this, we got a
zero point nine three. At this stage we
passed it through the activation
function so it got rounded up to one and
that's exactly the output that we wanted.
Cat is equal to one, so this structure
correctly identified that 1 0
should give you the output 1. Likewise
for the dog, the inputs were zero and one
and because one mul - was multiplied by
zero point zero six, we got a value of
zero point zero six here, which we passed
through the activation function, got
rounded down to zero, and that's exactly
the output that we wanted. So this
perceptron worked. It transformed 1 and 0
into 1 and 0 1 to 0 by multiplying by
these random weights. This is the
absolutely most basic type of neural
network that we can have. It worked
nicely, but what happens if a randomly
generated weights do not work? We will
need to correct them. And this procedure
is called backpropagation and we'll
study this in the next video.
