Hello everyone so today we are going to start
module 3 of this course and in the first lecture
of this module I will give an introduction
to eigenvalues and eigenvectors Why I am giving
this introduction to you because in this unit
we will focus on finding the eigenvalue and
eigenvectors using the numerical methods for
a given matrix Basically in real life problems
quite frequently we need to find out eigen
values or eigen vectors of a matrix for getting
some idea about the system and hence it is
very important to know about these two things
that is eigen values and eigen vectors for
a given matrix and how to apply a numerical
methods for finding eigenvalues and eigenvectors
of a given matrix
Let me define the eigenvalue and eigenvectors
So let A be a square matrix of order n So
it means this matrix say is the element of
the vector space Cm by n then a non-zero vector
X 
belongs to Rn or more generally you can write
it belongs to Cn is said to be an eigenvector
of A If AX equal to lambda X for some scalar
lambda notice this that I will said at X is
an eigenvector of A first of an it should
be a non-zero vector seconding it should be
satisfy this particular condition that is
AX equals to lambda X
Now let us talk about this scalar lambda 
Here lambda is called 
eigenvalue of A and hence we can say that
X is an eigenvector corresponding to the eigenvalue
lambda for example take a 2 by 2 matrix (2
1 2 3) so if I take a vector let us say (1
2) So this I can write it 4 times (1 2) So
this is my matrix A this is the eigenvector
X This is equals to lambda times X So it means
this particular vector (1 2) is an eigenvector
of this matrix A and 4 is an eigenvalue of
A and this eigenvector is an eigenvector corresponding
to eigenvalue 4
Now what this equation is telling what we
are having X is a vector and A is a matrix
So every matrix is a transformation basically
linear transformation So what we are doing
to be here applying a linear transformation
on a vector X and we are getting a scalar
change in the vector X scalar time change
So either the vector will expand or vector
will This will depend on the value of lambda
example for example in the earlier one I was
having initially vector (1 2) and after applying
the transformation it is becoming 4 times
(1 2) that is it is going to (4 8) So there
was a magnification in the vector of 4 times
So hence this is the another interpretation
of eigenvalue and eigenvector of a transformation
or matrix A
Now how to find eigenvalues and eigenvectors
of a given matrix? So methods for finding
eigenvalues and eigenvectors Now look at this
definition of eigenvectors Here I am saying
that X is a non-zero vector and it is said
to be an eigenvector of A if A times X equals
to lambda X So A into X equals to lambda X
can be written as AX minus lambda X equal
to 0 or this I can write this A minus lambda
I equals to 0 where I is the identity matrix
of the same order as A
Now from here you can see we are having here
system of homogeneous equation with where
we are having n equations and n unknown I
am saying that X is a non-zero vector So if
X is a non-zero vector means this system is
having non-zero solutions and if I talk that
this system is having non-zero solution it
means that is the null space of this particular
transformation A minus lambda I is having
dimension more than 0 Now if this is having
the non-zero solution it should be having
go and place then 1 less than n So rank of
this A minus lambda I should be less than
n it means 
determinant of A minus lambda I should be
0 because if rank is less than n
Now if I get the determinant of A minus lambda
I it will be a polynomial of n degree in lambda
and the zeros of that particular polynomial
are the eigenvalue of matrix A So using this
concept we can find the eigenvalues of a matrix
means what you need to do? You have to write
the matrix A minus lambda I you have to find
out the polynomial which is coming from the
determinant of A minus lambda I and solving
this equation now linear equation that is
a polynomial in of degree and in lambda equals
to 0 you will find the eigenvalues of A
Now as I told you by solving this equation
you can get the values of lambda for which
determinant of A minus lambda I equals to
0 and these values of lambda is called the
eigenvalues of A This particular polynomial
is called the characteristic polynomial of
A and hence eigenvalues are also called the
characteristic values of the given matrix
Now once you find out the eigenvalues let
us say eigenvalues are coming like this lambda1
lambda2 lambda n then (what we will) what
we need to do? We need to find out the eigenvectors
corresponding to each eigenvalue So eigenvector
corresponding to eigenvalue lambda equals
to lambda1 can be calculated just by solving
the homogeneous system of equations A minus
lambda1 IX equals to 0
Since we have choosed such a lambda1 for which
the system is having non-zero solution and
hence we will get the get a non-zero vector
X as a solution of this system and then non-zero
vector X will be the eigenvector corresponding
lambda equals to lambda1 Then similarly we
can find the eigenvalues eigenvectors corresponding
to other eigenvalues So this is the classic
way of finding eigenvalues and eigenvectors
of a given matrix So let us take an example
of it Let us consider tis 2 by 2 matrix So
first row is (3 2 7 -2) we need to find out
the uhh eigenvalues and eigenvectors of this
matrix So the characteristic polynomial of
A is determinant of A minus lambda I which
becomes 3 minus lambda minus 2 minus lambda
2 and 7 into minus 14 lambda square minus
20 and zero of this polynomial is 5 and minus
4 So hence the eigenvalue of this matrix is
5 and minus 4
Once we find the eigenvalues we need to calculate
eigenvectors So eigenvector corresponding
to lambda equals to 5 is given by A minus
5I into X equals to zero that is we got 2
equations one is minus 2X1 plus 2X2 equals
to 0 another one 7X1 minus 7X2 equals to 0
basically both are the same equation They
linearly dependent and the solution of this
equation is X1 equals to X2 So we choose X2
as 1 so X1 will be 1 Hence (1 1) is an eigenvector
corresponding eigenvalue lambda equals to
I Similarly we can calculate the eigenvector
corresponding to lambda equals to minus 4
and it is coming out 2 and minus 7 you can
verify that both of these eigenvectors with
respect to eigenvalue 5 and minus 4 satisfy
the relation X equals to lambda X
Now geometrically an n by n matrix A we are
multiplying it by n by n vector X So resulting
into another n by 1 vector Y equals to AX
Thus A can be considered as a transformation
matrix which is transforming a vector into
another vector of the same dimension In general
a matrix X on a vector by changing both its
magnitude and its direction however a matrix
may vectors by changing only their magnitude
and leaving their direction unchanged or possibly
reversing the direction These vectors are
the eigenvectors of the matrix okay
So if we are having a matrix it is a we are
applying this matrix on a set of vectors for
some of the vectors it will change magnitude
as well as direction however for certain vectors
what it will do if you are having n by n matrix
there will be a vectors n or less than n for
which just what it will do uhh linearly independent
vectors It will only change the magnitude
and those vectors will be the eigenvectors
of the matrix So in this way we can differentiate
or we can take out the eigenvectors of a matrix
from the set of vectors
So a matrix sets on an eigenvector by multiplying
its magnitude by a factor which is positive
if its direction is unchanged and negative
if its direction is reversed This factor is
eigenvalue associated with that eigenvector
So you can see X equals to lambda we are A
is acting on X and we are getting lambda X
what is lambda? Lambda is just a scalar so
what will happen if lambda is some positive
number? What will happen the direction of
the vector will never change because if it
vector is X1 X2 it will become lambda times
X1 lambda time X2
If lambda is negative number it will become
just uhh the direction of the vector becomes
in the reverse direction however magnitude
will certainly changed just take this beautiful
example here we are having this 2 by 2 matrix
A I am applying this matrix A on the set of
points So after applying this transformation
or this matrix on this set of points I am
getting these red points So blue points are
before transformation and red cluster of points
are after transformation So what is happening
it is changing This square say point to in
this shape
Now if I calculate the eigenvalues and eigenvectors
of this matrix Eigenvalues are coming 1 and
2 corresponding to 1 eigenvector is minus
0.71 and 0.7071 means it is something aligned
in the direction of Y equals to X and here
it is coming same so it is aligned in the
direction of Y equals to minus X sorry this
is the line in direction of y equal to minus
X this is aligned in the direction of direction
of Y equals to X and hence you can see what
are having this change is the maximum in the
direction of Y equals to X and what is the
scale of the change
This is scale of change is just double so
what we I want to say that in the direction
of eigenvector corresponding to largest eigenvalue
we are having the maximum change and change
is just multiple of that particular eigenvalue
and no change in the direction of this Y equals
to X because here eigenvalue is 1 So 1 into
that vector remains the same So here the role
of eigenvector corresponding to the biggest
eigenvalue becomes very important for analyzing
patterns or in when you are talking about
data analytics or patterns classification
on these areas So we will learn some numerical
methods to finding to find out the maximum
eigenvalue and corresponding eigenvector of
a given matrix in the coming lectures
Now as I told you the eigenvalues are the
roots of the uhh characteristic polynomials
and a polynomial can have repeated roots So
for example I can have lambda1 equals to lambda2
equals to up to lambda k So if that happens
the eigenvalue is said to be of algebraic
multiplicity k So what is algebraic multiplicity
of an eigenvalue? The algebraic multiplicity
is the number of times it is repeated for
example if I am having a matrix 3 by 3 order
matrix A and eigenvalue is (2 3 5) So hence
all the eigenvalues are having algebraic multiplicity
1 If I am having eigenvalue as (2 2 5) So
here eigenvalue 2 is having the algebraic
multiplicity 2 and 5 is having algebraic multiplicity
1 If I am having eigenvalue as (2 22) so 2
is repeating 3 times then algebraic multiplicity
of eigenvalue lambda equals to 2 is 3
Now for each distinct eigenvalue of matrix
A there will be correspond at least 1 eigenvector
which can be found by solving the appropriate
set of homogeneous equations Let k be the
algebraic multiplicity of eigenvalue lambda
If m is the number of linearly independent
eigenvectors please note that linearly independent
corresponding to eigenvalue lambda then 1
will be always lie between 1 to k means the
number of linearly independent eigenvectors
corresponding to an eigenvalue will be always
less than equals to the algebraic multiplicity
of the eigenvalue and this particular number
of linearly independent eigenvectors corresponding
given eigenvalue is called the geometric multiplicity
of the eigenvalue
So I want to say that geometric multiplicity
never exceed algebraic multiplicity like if
you take this example it is a 3 by 3 matrix
It is a upper triangular matrix It is having
characteristic equation lambda minus 2 cube
equals to 0 So hence it is having root 2 repeated
3 times algebraic multiplicity of eigenvalue
lambda equals 2 is 3 If I calculate the eigenvector
corresponding to lambda equals to 2 then I
am getting the eigenvector (1 0 0) and (0
0 1) So hence geometric multiplicity of eigenvalue
2 is only 2 whereas algebraic multiplicity
is 3
Let me tell you some properties of eigenvalues
and eigenvectors So the sum of eigenvalues
of a matrix equals to the trace of the matrix
So how can we define that trace of a matrix
Trace is the sum of the diagonal elements
of the matrix and hence sum of eigenvalues
equals to sum of diagonal elements of the
matrix and that you can see from the characteristic
equation very clearly The product of eigenvalues
of a matrix equals to the determinant of the
matrix Hence if an matrix is having a zero
eigenvalue one of the eigenvalues as zero
means the matrix is a singular matrix because
determinant is the product of eigenvalues
0 is coming there So product will be 0 Hence
determinant will be 0 The eigenvalues of an
upper or lower triangular matrix are the elements
of the main diagonal Similarly the eigenvalues
of a diagonal matrix are the elements of the
diagonal If lambda is an eigenvalue of A and
A is an invertible matrix then eigenvalue
of A inverse will be 1 upon lambda and what
will be having the same corresponding to same
eigenvectors
If lambda is an eigenvalue of A then k lambda
is an eigenvalue of matrix kA and this can
be in the proof is given very easily suppose
you are having a matrix AX equal to lambda
X now what you do multiply both side by A
inverse keep A inverse exists So A inverse
AX will become lambda times A inverse X and
from here I can write A inverse X equals to
1 upon lambda into X So if X is an eigenvector
of A corresponding to eigenvalue lambda then
X will also be an eigenvector of A inverse
corresponding to eigenvalue 1 upon lambda
So if lambda is an eigenvalue of A then lambda
race to power k will be the eigenvalue of
A race to power k for example if a 3 by 3
matrix is having eigenvalue as (5 10 20) then
I square of this matrix will be eigenvalue
is a 25 that is square of 5 100 and 400 moreover
it is very important result for the topic
which is I am going to introduce you in the
next lecture that is the similarity transformation
So this result tells us that eigenvectors
corresponding to distinct eigenvalues are
linearly independent moreover if we do not
have the distinct eigenvalues for a given
matrix and sum of the eigenvalue lambda is
having algebraic multiplicity k then the number
of linearly independent eigenvectors of A
associated with this eigenvalue lambda is
given by m and where m is thus n minus rank
A minus lambda I Now what is the diagonalization
of a matrix? So the eigenvalue and eigenvectors
of a matrix having a very important property
that is if a square n by n matrix A has n
linearly independent eigenvectors then it
is diagonalizable that is it can be decomposed
as A equals to PDP inverse where D is the
diagonal matrix containing the eigenvalues
of A along the diagonal so D can be written
as diagonal lambda1 lambda2 lambda n and P
is the matrix which is formed with the corresponding
eigenvectors writing the in the columns
For example if you take a matrix this (2 1
2 3) it is having eigenvalue 1 and 4 and corresponding
eigenvectors (1 -1) and (1 2) respectively
Then it can be diagonalized as (2 1 2 3) equals
to (1 1 -1 2) So it is my matrix P and you
can note here I have written this first eigenvector
as the first column of P second eigenvector
as the second column of P into D So D is a
diagonal matrix having the eigenvalue as the
diagonal entries only thing you have to note
down that first eigenvalue is 1 in the first
row So the eigenvector corresponding to 1
should be the first column
Similarly 4 is in the second row so the eigenvector
corresponding to 4 should be the in the second
column and then P inverse So if you multiply
these three maxtices the product of these
3 comes out to be this matrix and hence (you)
this particular property is very important
when you are talking about eigenvalues because
the eigenvalue of this P uhh A and the eigenvalues
of D will be same and since D is a diagonalized
matrix uhh diagonal matrix so eigenvalues
will be the elements in the main diagonal
Moreover we can use this property in many
other uhh ways like suppose you need to find
out A race to power m So this means PDP inverse
PDP inverse PDP inverse m times It is coming
out to be P into D race to power m into P
inverse where D is a diagonal matrix So D
race to power m can be calculated very easily
just by taking the power of lambda1 lambda2
uhh m power m and easily calculated and hence
we can calculate A race to power m in a very
say manner however all the matrices not have
this property that all the matrixes cannot
be writurn in the form PDP inverse
So in the next class what we will do we will
take some example where some of the matrix
I can write as PDP inverse but some of them
I cannot write I will tell you the condition
which is necessary for writing A equals to
PDP inverse I will tell you if I cannot write
A equals to PDP inverse then is there any
transformation other transformation which
is which able to uhh which makes this a factorized
into product of different matrices So now
I will stop myself and I will start from the
next lecture that is similarity transformation
thanking you
