Hello everyone uhh so welcome to the 4th lecture
of this module and today we are going to discuss
one more method for finding eigenvalues and
eigenvectors of a matrix So in the last lecture
we have discussed Jacobi method for finding
eigenvalues for a symmetric matrix and there
we have uhh we have use the similarity transformation
in such a way the a similar matrix has uhh
we a similar matrix found corresponding to
the given matrix and that similar matrix is
just like uhh just a diagonal matrix where
the eigenvalues are the diagonal elements
like Jacobi method was restrict up to symmetric
matrix only today we are going to discuss
a method which is applicable to any square
matrix however again we are having some conditions
to apply power method which is we are going
to discuss today and we will discuss about
those conditions
So first of all let A be a square matrix of
order n the eigenvalues are calculated just
by solving the characteristic equation which
is given as lambda raze to power n plus some
constant Cn minus 1 lambda race to power n
minus 1 and so on So it is a n degree polynomial
in lambda and roots zeros of this polynomial
will give you the eigenvalues Let us say eigenvalues
are lambda1 lambda2 lambda n where some of
them may be equal or repeated we will say
that lambda1 is a dominant eigenvalue of A
if this condition is satisfy means the absolute
value of lambda1 is greater than rest of the
absolute values of rest of the eigenvalues
that is mod of lambda1 greater than mod of
lambda i for i equals to 2 3 up to n and if
this condition old lambda1 is the dominant
eigenvalue and the corresponding eigenvector
is called dominant eigenvector of A
So for example if I take this 2 by 2 matrix
So the characteristic polynomial of this matrix
is lambda square plus 3 lambda plus 2 which
is having zeros as lambda1 equals to minus
1 and lambda2 equals to minus 2 So the eigenvalues
are -1 and -2 for this matrix Now if we see
that absolute value of -2 is greater than
absolute value of minus 1 Hence -2 is the
dominant eigenvalue of this matrix and corresponding
eigenvector is (3 1)
Now what are the conditions to apply power
method First of all eigenvalues can be arranged
in the following way that is the dominant
eigenvalue should be there for the matrix
and there should not be any repetition of
dominant eigenvalue for example if we are
having a 3 by 3 matrix there should be 1 dominant
eigenvalue which is not equals to others for
example if A is 3 by 3 and we are having eigenvalues
minus 5 3 and 2 Then here minus 5 is clearly
dominant eigenvalue but if we are having eigenvalues
like -5 5 and 4 Here we cannot apply power
method as such and we we will not be able
to find out the eigenvalue because here the
dominant eigenvalue is -5 as well as 5 and
it is repeated So here lambda1 is not clear
strictly greater than the rest of the eigenvalue
in terms of absolute value The second condition
is which is again very important that the
matrix A should have a linearly independent
eigenvectors It means A should be similar
to A diagonal matrix if we talk in terms of
similarity transformation
So with these two conditions let us derive
the power method So let A is n by n matrix
and it is having eigenvalues lambda1 which
is strictly greater than in terms of absolute
value to rest of the eigenvalues moreover
we are having eigenvector as V1 corresponding
to lambda1 V2 corresponding to lambda2 and
Vn corresponding to lambda n and here we are
assuming that V1 V2 Vn are linearly independent
So if these vectors are linearly independent
then any vector V from the vector space Rn
can be written as the linear combination of
these eigenvectors So if V belongs to Rn or
if matrix is from the from the complex number
and from the Cn So if we form Rn then we can
write V equals to C1V1 plus C2V2 plus Cn Vn
Here C1 C2Cn are scalars
Now if I multiply by matrix A in this equation
I will get in the left hand side A into V
and then C1A into V1 plus C2A into V2 plus
Cn A into Vn as we know that V1 is an eigenvector
corresponding eigenvalue lambda1 for the matrix
A So I can write C1 lambda1V1 Similarly this
term I can write lambda2 V2 plus Cn lambda
n Vn or if I take lambda1 common from the
right hand side I can write C1V1 plus C2 into
lambda2 upon lambda1 into V2 So this is equals
to AV If I multiply one more time by the matrix
A in this equation I will get A square V in
the left hand side and this will become 
so C1V1 plus C2 and the square of lambda2
upon lambda1 into V2 plus Cn square of this
ratio term into Vn or if I continue by multiplying
A again A again let us say I multiply k time
it will become A race to power k V equals
to lambda1 race to power k into C1V1 plus
C2 lambda2 upon lambda1 race to power k 
and finally the last term will become
Now look at this equation here we are having
in the this term lambda2 upon lambda1 Similarly
in the next term we will be having lambda3
upon lambda1 and so on Here our assumption
is that lambda1 is the dominant eigenvalue
It means that this particular term lambda2
upon lambda1 will be less than 1 Similarly
lambda3 upon lambda1 will be less than 1 and
up to lambda n upon lambda1 which is again
place then 1 So when k is tending to infinity
my Ak V will become lambda1 race to power
k into C1V1 It means I can find out lambda1
as limit k tending to infinity by this ratio
and here R is the component of the vector
this particular vectors having the highest
value in magnitude So this will give me dominant
eigenvalue and since we are using various
powers of A that is why we are saying it the
power method and from this equation it is
clear that if lambda1 is the eigenvalue that
is the dominant eigenvalue then the corresponding
eigenvector will be V1 So with this I can
talk about the conversions of this power method
So if lambda2 upon lambda11 and absolute of
this ratio term is less than 1 the rate of
conversion is fast moreover whatever will
be the means if it is quite small then 1 the
method we converse faster if it is close to
1 this ratio term the method will converge
slowly as we know that power method is an
iterative method because each time we are
making we are starting with the initial solution
V0 then we are finding V1 as A of V1 then
I V2 will be A of V2 V3 will be A of V2 and
so on until unless it will not converge So
there should be some stopping criteria and
the stopping condition is if in the two successive
iterations lambda is less than a given threshold
okay in terms of 10 race to power -3 or -5
whatever accuracy you want in your method
or the maximum component in 2 successive vector
Vk plus 1 and Vk and difference of this which
is having the maximum value is less than a
given threshold Moreover to control the round
off error the vector is normalize before pre-multiplying
by A so that the largest element remains unity
for example if you start with 1111 and after
multiplying with A you are getting let us
say some vector 2 34 So what I will do I will
normalize it in such a way that the biggest
component of this V2 vector that is 4 should
become 1 so that vector will become 2 by 4
3 by 4 and 1
If we talk about eigenvector so we start with
V0 as the initial vector and the condition
is that this V0 should not be orthogonal to
vector V1 So and it should be a non-zero vector
obviously because we are finally converge
in this particular vector is conversing to
the eigenvector So yk plus 1 equals to A of
Vk then I will find out Vk plus 1 as yk plus
1 upon mk plus 1 and as I told you where mk
plus 1 is the largest element in yk plus 1
in magnitude So in this case lambda1 will
be limit k tending to infinity yk plus 1 r
upon Vk r finally Vk plus 1 will be the required
eigenvector corresponding to lambda1
Let us take an example of this method just
consider this 3 by 3 matrix and let the initial
column vector be 111 So first of all I will
find out V1 uhh V1 will be A into V0 A is
this 3 by 3 matrix V0 is 1 11 this column
vector after multiplying I am getting another
column vector which is V1 2-1 and 0 Now what
I will do first of all I will find out from
V1 to y1 and y1 will be I will see which is
the biggest component in this vector in terms
of a float value and here it is 1 So I will
divide this V1 by 2 So y1 will become 1 by
2 1 0.5 0 So here I can say that in tis iteration
my eigenvalue is 2 and the eigenvector is
1 -0.5 0 Then I will calculate V2 V2 will
be A into y1 and A into y1 when I will calculate
it it will become 3.5 minus 4.5 again I will
divide this vector by 4 so that this term
will become 1 -0.875 1 and -0.125 So here
in this iteration eigenvalue the approximation
of eigenvalue is minus 4 So just look in first
iteration it was 2 in the second iteration
it is coming -4 and similarly we are getting
a deviation is eigenvector In the third iteration
eigenvalue becomes 6.125 that is approximation
and y3 becomes -0.918 1 and -0.1837
So we are not getting any sign of convergence
so far in 3 iterations however if we go up
to 14 iterations what we found that the approximate
of eigenvalue which I am getting in 14th iteration
minus which I am getting in 13th iteration
that absolute difference between these two
is less than 10 race to power -4 and hence
the power method converge to eigenvalue 5.4773
which is the dominant eigenvalue of the given
matrix and the corresponding eigenvector becomes
-0.4037 1 and -0.2233
There are some disadvantages or limitations
of this method The first one is if the initial
column vector V0 is an eigenvector of A other
than the dominant eigenvector then the method
will fail since the iteration will converge
to wrong eigenvalue moreover the speed of
convergence depends on the ratio magnitude
of dominant eigenvalue lambda1 upon magnitude
of the next largest eigenvalue If the ratio
is small the method will converge slowly The
power method only gives one dominant eigenvalue
at a time okay I will tell you how can we
find out other eigenvalues using this method
So as I told you there are some limitations
when I told you the assumption for applying
this method I told you that there should be
a dominant eigenvalue if this is not the case
what will happen whether the method will converge
or not Let us see it with an example So if
I take a 2 by 2 matrix let us say 1 1 0-1
Let us find out the eigenvalue of this method
eigenvalue of this matrix using power method
So let me start with a initial vector V0 which
is 11 So V1 will become 1 1 0-1 that is my
matrix A into V0 11 So 1 plus 1 2 and -1
Now I will calculate 2 V2 will become A of
V1 So it is 1 1 0 -1 into 2 minus 1 So 2 minus
1 will become 1 and it become 11 again then
if I will calculate V3 V3 will come 2 minus
1 V4 will come 1 1 and so on So my method
will stuck here in these two vectors in either
I will get for the odd iterations of V I will
get 2 -1 for the even iterations like 2 46
I will get 11 and I will it will never converge
Why it is happening This is clear from here
if you see the eigenvalue of this matrix it
is an upper triangular matrix and here eigenvalues
are 1 and -1 and the region of this oscillation
is very simple that the matrix is not having
the dominant eigenvalue that is why the condition
that the matrix should have dominant eigenvalue
is quite important for applying the power
method up to now we have seen that using the
power method we can calculate up only dominant
eigenvalues and corresponding eigenvector
suppose I want to calculate other eigenvalues
also
So we can modify this power method in such
a way that we shift the dominant eigenvalue
to 0 in a new matrix such that the second
2 dominant eigenvalue become the dominant
for example if you are having eigenvalues
lambda1 lambda2 lambda3 if lambda1 is dominant
what we will do we will shift tis lambda1
to zero in some other matrix such that the
lambda2 becomes the dominant and then in this
new matrix we will apply the power method
So this method is called method of deflation
and it is based on deflation theorem So how
it works So once you calculate the dominant
eigen pair that is lambda1 V1 of a matrix
as A you will calculate the or you want to
calculate lambda2 Here so I will take an example
of symmetric matrix but it can be generalized
for any other matrix also So if A is a symmetric
matrix then it can be prove that if U1 is
V1 upon mod of V1 then A1 is A minus lambda1
U1 U1 transpose has eigenvalues 0 lambda2
lambda3 lambda n and the eigenvector of A1
will be the same as of A so this is one of
the result of deflation theorem
So here we are saying that if A is n cross
n matrix having eigenvalue lambda1 lambda2
lambda n and corresponding eigenvectors are
V1 V2 Vn Now and also we are assuming that
lambda1 is the dominant eigenvalue and the
corresponding eigenvector to this V1 that
is a dominant eigenvector is V1 Now I am saying
if A is a symmetric matrix I can calculate
a new matrix A1 or let us say it B which is
A minus lambda into U into U transpose where
U is the unit vector in the direction of V1
then this matrix B will be having the eigenvalues
0 lambda2 lambda3 up to lambda n and the eigenvectors
will remain same like V2 V3 Vn for this new
matrix B so what we can do suppose using the
power method on this matrix A we calculate
the dominant eigenvalue and corresponding
eigenvector that is lambda1 and V1 and here
this is my lambda1 is the dominant eigenvalue
in this result
So so I calculate these two what I will do
I will apply this transformation I will get
a new matrix B and again I will apply the
power method on B so that I can calculate
the next eigenvalue to the dominant that is
lambda2 and corresponding eigenvector that
is the dominant eigenvalue of B will be the
next two dominant eigenvalue of A and how
we are getting this result Suppose A is a
symmetric matrix so I want a new matrix B
which is something A minus lambda V into X
transpose So I am taking a vector X if I multiply
this vector X in this second term of the right
hand side of this equation I get a new matrix
B which is having the one of the eigenvalue
as 0 If lambda is the eigenvalue of A
Now how If V1 is the eigenvector corresponding
lambda of A so I will be having B of V1 equals
to A of V1 minus lambda V1 X transpose V1
and as I told you lambda1 and V1 are the dominant
pair of matrix A So AV1 can be written as
lambda1 V1 and then it will become lambda1
V1 minus lambda1 V1 X transpose V1 So it will
become 1 minus X transpose V1 Now how to choose
this vector X such that one of the eigenvalue
of B should be 0 and the corresponding eigenvectors
would be V1 So here if this term become 0
then what I can have If this term is 0 and
I choose X in such a way then BV1 will become
lambda1 V1 into 0 that is BV1 equals to 0
means one of eigenvalue corresponding for
which this is the eigenvector should be 0
So for a symmetric matrix I can take this
X as this one so that my this becomes U and
lambda1 UU transpose transpose this becomes
the deflation transformation
Let us check this with one of the example
again I am taking a 2 by 2 matrix and now
I question is find all the eigenvalues of
this matrix using power method together with
method of deflation So it is a 2 by 2 matrix
so there will be only 2 eigenvalues uhh one
of the eigenvalue that is the dominant eigenvalue
we can calculate using the power method and
the other one we will use the method of deflation
in power method So let us start with 1 1 So
using the power method and after going up
to 10th iteration what I am getting that eigenvalues
eigenvalue that is the dominant eigenvalue
is converging to 9and the corresponding eigenvector
is converging to minus half 1 So hence one
of the eigen value of this matric the bigger
one in terms of absolute value is 9 and the
corresponding eigenvector is minus 0.51
Now I apply this transformation deflation
so A will become A minus uhh B will become
or A1 I have written A1 will become A minus
lambda1 UU transpose So after applying this
my A1 is coming 4 upon 5 into (4 2 2 1) Now
again I will apply and from the method deflation
theorem one of the eigenvalue of this matrix
will be 0 and the corresponding eigenvector
will be V1 that is minus 0.5 and 1 which is
corresponding to 9 or A So by applying the
power method again on this new matrix A1 starting
with 11 we get X2 as this 11 upon 4.8 into
1 upon 1 by 2 and after going this way we
will see that the method is converging to
4 as the eigenvalue dominant eigenvalue of
A1 and corresponding eigenvector as 1 and
0.5
So hence this lambda2 equals to 4 is the dominant
eigenvalue of A1 but it is the other eigenvalue
that is the second eigenvalue of A and the
corresponding eigenvector is 1 with and 0.5
as the two components So hence using method
of deflation with power method we can calculate
all the eigenvalues of a given matrix So in
this lecture we learn how to use power method
for finding the dominant eigenvalue and corresponding
eigenvector of a given matrix later on we
have seen method of deflation if we apply
together with power method we can calculate
other eigenvalues also those are not dominant
of the given matrix Thank you very much
