So, we saw a couple of prerequisites and we
are all set to go ahead and thenah make our
required observations ah. Let us go very very
slowly ah. I will help you all recollect what
has happened so far and then try to connect
different pieces of the puzzle.
So, what does one observeof this matrix multiplication
process right? So, we observe that no matter
what vector we choose , irrespective of what
vector we choose remember the screen cast
that I did ofthe python programming code.
I took different vectors. No matter what vector
I chose, I landed up with the same vector
right.
We all converged we observe that every single
vector, no matter where you start from the
repeated application of the matrix results
in the very same vector . By very same vector,
I mean the same direction right. This we observe.
Why did this happen? What makes thisprocess
of applying the matrix repeatedly on a vector
and scaling it down of course, scaling it
down is to ensure that the numbers do not
become big that is all, nothing else. As you
keep applying the matrix onany given random
vector, it always goes to the same point inr
2 plane; r 2 is that two dimensional plane
correct.
Why is this happening? What is the physics
behind it? What is the logic behind it? Let
us unravel that slowly. So, let us recollect
some basics from ouragain high school mathematics.
Throughout our discussion we are not going
to use anythinghi-fi. All we are going to
use is some very basic matrix theory.
I am sure you all heard of eigen values and
eigenvectors right. So, let us recollect that.
So, given a matrix we know there is something
called an eigenvector and eigen value. Now,
what is that? Let me help you recollect it
and eigenvector is something of a matrix A
eigenvectorv is defined as something that
simply gets scaled up by a lambda factor.
Then lambda is called an eigen value and the
v is called an eigenvector correct ok.
So, this is a right time for you to open let
us say Wikipedia or any other online reference
and then refresh your basics of eigenvalues
and eigenvectors. You only need this definition
that A of v is equal to lambda v. This is
first thing that we need to know and second
thing that we need to know is for a 2 cross
2 matrix, let us say a 2 cross 2 matrix A;
there are 2 eigenvectors not always, but mostly
you will always have 2 eigenvectors ok. And
these eigenvectors are independent. They are
independent . What do I mean by that? By that
I mean you take any vector any vector of your
choice .
Let us say any vector z you can always write
z as a linear combination of v 1 plus some
beta times v 2 because that they are what
is called linearly independent you can always
write any vector as the linear combination
of v 1 and v 2. If you do not know these things,
you probably shouldbrush up your basics. So,
this is the third one.
First one is the definition of Eigen values
and eigenvectors, second one is given 2 cross
2 matrices is matrix. There is always 2eigenvectors
and they are actually linearly independent
ah. What do you mean by linearly independent?
Two vectors that are linearly independent
in r two given any point in r 2 that point
can be written as the linear combination of
these 2 eigenvectors .
So, this point is z you can always write z
as alpha times v 1 and beta times v 2. this
is the basics of matrix theory. I am sure
all of you are familiar, if not please take
a pause and take a look at it. You need not
know the reasoning behind all these things,
you just need to recollect these things that
should be enough ok.
So, let us go further now ok. So, please revise
now before going any further. You should know
what are eigenvectors and eigenvalues. I am
not going to apply them in a in anyah not
so obvious manner. Every single application
of this concept will be pretty straightforward.
So, I hope you have recollected what is eigenvector
and what is an eigenvalue and I proceed further
ok. Now how and why of eigenvectors and eigenvalues
here? So, what did we do in our programming
screen cast? What did we observe of this matrix
multiplication? Whenever a matrix a acts on
any vector v any vector vplease observe that
A is a matrix v is some random vector v , then
you can always write v as a linear combination
of lambda 1 times v 1 plus lambda 2 times
v 2. This is always possible right. We just
now saw in that in our previous prerequisite.
This is always possible; let us note this
ok.
Next so, this v 1, v 2 are eigenvectors and
lambda 1 lambda 2 are eigenvalues. We observed
it already ok, I am just helping you recollect
it by saying it once more ok so far so good.
So, all I am saying here is any given vector
v if a matrix A is applied on it, then you
can always see it as A being applied on a
linear combination ofeigenvectors v 1 and
v 2 no matter what we you choose. Now what
is this equal to? This is equal to A is A
matrix. Its application on a scalar lambda
1 times a vector v 1. you can always pull
out the scalar here as; you can see you can
pull out the scalar here correct.
So, you can write it as lambda 1 A times v
1 and lambda 2 A times v 2. But then, but
thenobserve carefully, what are v 1 and v
2 ? v 1 and v 2 are eigenvectors and and so,
so, what if they are eigenvectors? If they
are eigenvectors, you can further write this
A of v 1 as lambda 1 times v 1 right; A of
v 1 is lambda 1 times v 1 A of v 2 is lambda
2 times v 2 and you finally, get this correct
ok .
As I continue this process, look at my previous
slide as I continue this process I apply A
again on this, I continue to apply A again
on this . What do I get? I repeatedly apply
A ok that is equivalent to me applying A k
times on v. So, I am talking a lot of things
sort of very quickly. I suggest that youtake
a pause and then take a look at what I am
saying ok all right. So, I am applyingah A
k times on v which gives me lambda 1 to the
k times v 1, why? Pretty obvious look at the
previous slide if you are confused look at
this, understand this carefully and you will
understand what I am doing here.
This gives me lambda 1 to the k times v 1
plus lambda 2 to the k times v 2 correct perfect
so far so good. Absolutely no confusion so
far right. Observe carefully this lambda 1
let me assume is greater than lambda 2 all
right. This always holds good. Eigenvalues
are most of the times distinct when they are
distinct one of them is greater than the other
one.
So, when one of them is greater than the other
one, what happens? Lambda 1 is basically the
amplitude right lambda 1 to the power of k;
if lambda 1 is somelet us say 2 a, number
greater than 1, then lambda 1 to the power
of k for a huge k will be a huge value right.
This is what we call as amplitude for the
vector v 1. When you multiply lambda 1 to
the k 2 v 1 it sort of scales v 1 up , lambda
1 to the k being a big number when multiplied
to a vector v 1 it just makes this vector
shoot away from origin right. We have discussed
this already.
So, think about it for a minute. v 1 simply
simply signifies the direction and this product
tells us thefinal vector which is extremely
scaled right. The value the existing v 1 is
get is getting pushed by this value lambda
1 to the k that is what this means ok.
Let us note something here. Let us observe
this carefully. Let us take these two numbers
2 and 3 right just just plane simple 2 number
2 number 3 and look at this 3 is one more
than 2 correct as simple as that 3 is 1 more
than 1 right which is like saying 3 is 50
percent more than 2 correct.
But then when you square it 2 square gives
you four 3 square gives you 9 and then what
happens? This 9 is more than twice of 4. When
you take 2 numbers A and B,if B is greater
than A, if you look at their proportion by
what factor it is greater, you observe that
this number is 50 percent greater than this
number . But when you square them, you observe
that it turns out to be twice as much as this
number. As you continue this way you will
observe that look at this what happened.
If you cube it, you get 8 and 27 ; now that
is surprisingly more than 3 times. So, this
is like saying let me give you a very nice
fictitious example ah. Look at your bank balance
look at my bank balance. Assume your bank
balance is 2 lakhs and my bank balance is
3 lakhs . Let make you feel happy by making
you rich. Assume your bank balance is 3 lakhs
and my bank balance is 2 lakhs all rightwhich
is like you are just 1 lakh richer than me.
So, assume God comes and cubes our bank balance,
he cubes. So, my bank balance was2 2 he makes,
it 2 cube and your bank balance was 3 lakhs,
he makes it 3 cube. So, initially you were
just 50 percent richer than me, but now you
become more than thrice richer than me right.
So, this although God came and cubed me as
well as you, he did this he gave the same
gift worth to me and you depending upon what
was the number that we had we ended up having
a bigger number. A person who had more, now
has a lot more. I think you got the intuition.
So, now as we keep going on further, this
was more than thrice. We observed we keep
doing this. Let us say we empower it by 100
then, we observe something really startling.
This 3 to the 100 is several several several
force bigger than 2 to the 100. So, big that
let us observe what happens. So, big it several
force big that so much more than this that
if you look at the ratio it is close to 0,
why?
2 to the 100 divided by 3 to the 100 as you
can see is 2 by 3 whole to the power of 100.
2 by 3 is a number smaller than 1 and you
are empowering it to the number 100 which
is a very big number. You take as number less
than 1 and keep multiplying it to itself it
will quickly go to 0 you see, that is what
is happening here. Take a minutes pause and
observe what exactly we explained in this
slide look at the previous slide right.
We are talking aboutthe multiplication of
a big number namely lambda 1 to the k to v
1 and lambda 1 is greater than lambda 2. What
is the connection between this and what we
discussed here? What is the connection between
this slide and this slide? Take a minute,
I repeat stare at this slide. See what is
happening here stare at this slide, what did
we just now say and collectively we can say
something in the next slide. This is the right
time to pause and think about and guarantee
yourself that you understand this slide as
well as this slide.
Now, let us go to the next slide if you are
through with these 2 slides. It is so big
that if the ratio is 0, we saw that which
implies that 3 to the 100 is several folds
bigger than 2 to the 100 ok. I am just just
stating the same thing repeatedly. So, what?
So, we can make a biginference now .
We observe that when you empower A when you
repeatedly apply A on any random vector v,
you can always write this as lambda 1 to the
k v 1 plus lambda 2 to the k v 2. What happens?
This is amplitude and this is the direction
we discussed that lambda 1 is greater than
lambda 2 which implies lambda 1 to the k is
very greater than lambda 2 to the k; if k
is big. We saw 2 and 3 example and k was 100.
It was huge huge so much that the bigger one
simply completely dominates or the smaller
one so much that the smaller one is negligible
in front of the bigger one so much. So, that
the ratio goes to 0 correct ok.
So, now this implies that A to the k of v
is lambda 1 to the k v 1 is a very big quantity,
why? That is because lambda 1 is greater than
lambda 2 and this results in lambda 1 to the
k being very very greater than lambda 2 to
the k. This is a bigger entity, this is too
big, this is small . Small compared to what?
Small in comparison to the big thing that
is sitting here I am sorry it is not the v
1 which is way; it is this entire this entire
thing that is big is what I mean here when
I say big. So, entire thing is small small
in amplitude is all I am saying ok. Let us
go next.
So, we saw that A to the k of v is so much
and this one is really huge. This one is really
small. When I say really small, it is comparatively
small right ok. So, now, when you take a big
vector and add it to the small vector, what
do you get? Do you recollect? Do you do you
see the bells ringing in your mind? We saw
prerequisite right. We saw that whenever a
big vector is added to a small vector, it
will simply be in the direction of the bigger
vector right which means my A A to the k of
v will result in lambda 1 to the k of v 1
and you can simply ignore this factor here
ok, perfect.
So, what do we conclude? We conclude, it is
in the direction of v 1 and please note this
v was a random vector. Let me write that down.
This is a most important observation, it was
a random vector and no matter what you chose
for v, no matter what you choose for v its
was a random vector, no matter what we chose
for v; you always ended up with v 1. What
is v 1? v 1 is the eigenvector corresponding
to the highest Eigen value. If lambda 1 is
greater than lambda 2, then I take that corresponding
Eigen vector and this is independent of this
v and it is something to do with the matrix
that you take.
So, whatever vector you take, you always end
up with the eigenvector. Do you see why? I
just gave the explanation. Again this is the
right time to pause and then understand what,
I just now said and that completes the proof
for the fact that wheneveryou take a matrix
as screen cast if youif you can recollect,
we did a screen cast of our programming where
we took a matrix 1 2 3 4 and applied it on
different vectors. It was going to the same
vector, why? This is the reason
