Hello people, welcome back to my channel. The topic for
today's video is the decision boundary of
a neural network. So in the last video that
we saw the Matrix representations and many
other different forms that we may use in deep
learning. So in today's video, let's see how
a decision boundary looks like in a neural
network. Now we have learnt in different algorithms
including decision trees and the linear regression
and many other it is we have a Specific kind
of decision boundary for each of the data
set and it depends upon how the input distribution
is. So in a neural network, there is no particular
kind of decision boundary that is in general
for each and every instance that which it
incurs instead as in when how the data comes
it will learn the decision boundary as it
is, so say for example now since I have this
axis and say if I have some instances say
Like this and I have some more instances like
this. There it is. This is a two-class problem.
So now what you can see is the distribution
of the data. That is homoscedastic means that
is uniform across the two separations. Now
if I ask you like whether I can draw a decision
boundary that can well separate these two
different classes so I can essentially say
like I can draw something a decision boundary
like this where the equation of this is equal
to Y is equal to of form X So where y minus
X is less than 0 on this side and Y minus
X is greater than 0 for this other gloves.
So essentially in two Dimension, you can see
this as a line, but in three dimensions, you
can assume this as a plane that is separating
these two so whatever instance lies on the
upper side that belongs to this Cross or the
black class and whichever Falls below this
line that belongs to blue class now say for
instance. We have some another setup. So in
the real world, you don't have the data set
all in a linear formation. So there can be
non-linearity also, so that's why we introduced
the nonlinear part. So we'll come into that
a bit later. So now say our distribution,
is say like we have now the point's lead in
this way and we have some more points which
are The same of this class
Now, this class is somewhat homoscedastic
means it is uniform but the lower class which
is the that is for the blue. It's not uniform
but there is some kind of particular curvature
that is following. So essentially We cannot
put a straight line that passes through or
that separates these two classes. So essentially
what we can have is we can have a decision
boundary something like this. Now, this particular
decision boundary is not Linear, so it is
a parabolic curve. So we are mainly doing
the curve fitting that is of the form Y is
equal to x square. So essentially you can
see that this particular looks like a ball
shape in three dimensions. So where for the
cost function for linear regression. You have
the slope and the intercept for a neural network.
You have the weights as well as the biases.
So the neural network will also learn this
kind of decision boundary and then you have
some another set up say you have some points
which are there inside this And say you have
some points each other outside say just take
the triangle class.
This is how the data distribution is. Now.
You cannot draw a straight line which separates
these two neither. You can draw a parabolic
curve fitting. So what you can essentially
do is you can draw a circle. So which is of
the form x square plus y square is equal to
a square for some constant and for the classes
which lie inside x square plus y Square minus
a square is less than 0 and for outside. This
equation becomes X. Plus v Square minus a
square is for this inside less than outside.
It is greater than 0 that is four points lying
on the circle for that you have this equation
of a circle and four points that lie inside
the radius or this particular value is less
than 0 and for outside is greater than zero.
So now you can see different types of decision
boundaries, are there across different. Algorithm.
So you need to make your neural network learn.
These are all decision boundaries. And so
if your equation if you remember and that
is we have that is the linear part when we
compute W1 X1 plus W2 X2 up till you have
wnxn in so this was the linear part that we
were Computing in this particular unit. So
then we have some nonlinear function. So mainly
we would use sigmoid function so that it transforms
into some essential formation. So now say
you are you willing to work is just learning
only this particular linear function, but
then now every and each and every time it
is classifying your instances in this way
and if some distribution comes like this,
then it is unable to classify this, why because
it is just only learning your linear part.
So that's why you need to have a nonlinear
function. So So we drew some equations x 1
x 2 then we had the bias as well that is for
adjustment. And then we compute the output
that is the estimate so having a linear function
and nonlinear function. Both are an indispensable
part of a neural network, so you can't just
take away the nonlinear part from this unit.
Like the human brain, so if you have only
the right brain and if you don't have the
left bring them that becomes difficult for
you to survive. So similar is the case with
the neuron it won't survive through but then
it will collapse immediately after a few competitions
or neither. It will not do any computations
at all. So I'll just give you one more example
say we have some mathematical equations like
we have said W is represented in terms of
a x. Then say you have X in terms of v y and
Y in terms of c z now. I'm asking you to represent
W in terms of Z. So how would you do that?
So first you will take this equation in place
of X, you would replace it by b y and in place
of Y you would represent by z. So assume all
these ABC's are constant so say just consider
this is number 3 into 2 into 1 that becomes
6, so w is nothing but six times of Z. So
this is a linear function now in the real
world say our equation is of the form Z is
equal to W 1 x 1 plus W 2 x Cube W 3 x 4 And
so on W NX n if you have these kinds of representations
or this is not a monomial but this is a polynomial.
So if your neural network is not able to learn
these things then it becomes difficult for
you to transform or train your neural network
for each and every scenario, then you do change
that architecture which we have seen and which
essentially affects your generalization of
a neural network. So learning these kinds
of functions becomes essential. So before
the era of deep learning if these things were
coming into your equation then how they would
treat is by using something called feature
engineering. So we're what did is they try
to replace these coefficients which were not
linear in the input. They try to pass it to
some function to make it linear and then they
would put into your function but in the current
deep learning techniques or strategies, we
don't do feature learning. He's in learning
is not our feature engineering is not a part
of current deep learning trends. So all the
inputs are considered as linear and then we
do it in some nonlinear function by using
sigmoid or other functions like rail you are
and it's so what we mainly see is we have
some input layer and we have the output layer
and then we have n number of hidden layers
so up till one to of up till n hidden layers.
Is so it can be up till any number for the
complexity. Now if your neural network is
just only learning or just taking the inputs
and just multiplying it with the weight Vector
then what is the purpose of your neural network?
It is not merely learning anything just it
is multiplying and there is just adding up
with all the other weights so that this just
you can do with your simple math also, but
in order to learn the non-linearity present
in particular data set how distribution is
how it would change in a particular period,
you need to have this non-linearity part also,
so it becomes an indispensable part of your
neural network in learning so that it can
learn each and every decision boundary which
is there and which is offered by other machine
learning algorithms. That is by an SVM or
if you have some other curves parabolic curve
like linear regression decision trees and
many other so well, that was all regarding
the decision boundaries of neural networks.
So hope you guys enjoyed this video, if you
got educated by this video Please do like,
share, comment. And if you're new to this
Channel, please consider subscribing. Thank
you very much for watching this video.
