So, welcome back this is lecture number 46
and today we will continue with Eigenvalues
and Eigenvectors another very important topic
in linear algebra.
So, we will go through the introduction of
these eigenvalues and eigenvectors and also
their geometrical interpretation and, then
some simple examples to evaluate eigenvalues
and eigenvectors.
So, here what are the eigenvalues and eigenvectors
of a matrix; here let us consider this simple
matrix here A a 2 by 2 matrix given by this
3 minus 2 1 0. And, then we consider these
two vectors one is this u which is minus 1
0 and another one is v which is 2 1. And,
with this if we compute this product here
A and u so, here we have this A and then we
have this u. So, this product will be A minus
3 and then minus 2 that will give minus 5
and then here minus 1. So, the second component
will be minus 1. So, we have this result minus
5 minus 1, but if we do this product with
this vector v then what will happen?
So, A times v now so, A and this is v here
2 1 then what we will get so, this is 6 and
minus 2 4 and then here we will get 2. So
now, this result of this product is 4 a 2;
what is interesting here now this 4 2 is nothing,
but the 2 times of the vector here 2 1. So,
what we observed now that this A times v in
this case is this 2 times v. So, this product
here is still giving us v, but it is just
the 2 times here one number has come in front
of this v. So, this length of this vector
which was v here after multiplication this
has increased or this has got double now.
So, that is exactly the point which will take
us to this introduction to eigenvalues and
eigenvectors.
So, if we look at this geometrically what
is happening. So, here we have this x axis
and y axis. So, this vector v which is given
here as is exactly this one and then the Av
after this product it is just the 2 times.
So, that is the vector this width length double
of this length of this v. So, we have this
vector Av; what is interesting that this Av
and v they have the same direction and their
magnitudes are different. So, here now it
is just the double of this earlier vector
v after the multiplication, but in this case
of this u vector this was the vector u and
this A times u has become this one.
So, we do not have such relation that after
multiplication the vector direction remains
the same, we have a completely different vector
now Au. So, our interest is not exactly this
u with this A, our interest is for such vectors
whose multiplication with that matrix does
not change its direction its a parallel to
the original vector v. And, that is what we
are looking for and these are called actually
the eigenvectors and this number which has
come here that will be called as eigenvalues.
So, that is the topic of today’s lecture
and we will go little more into the detail
now about these eigenvalues and eigenvectors.
So, the definition the mathematical formal
definition here: let A be any square matrix,
the entries can be real or the entries of
the matrix A can be complex. A scalar lambda
is called eigenvalue of A if there exists
nonzero vector yeah, that is also important
here this nonzero; vector if there exists
a nonzero vector such that such that this
Ax is equal to lambda x. So, exactly it is
a parallel to what we have just seen in previous
examples. So, if we have such a vector x whose
multiplication with this given matrix a Ax
is nothing, but the lambda time x and this
lambda is some scalar some real number or
a complex number.
So, here this Ax is equal to lambda x that
is a very important equation which we will
be talking about today. So, that is the definition
now of this eigenvector this x is called the
eigenvector and this lambda is called the
eigenvalue. So, this is the eigenvector and
this has to be always nonzero otherwise, the
0 will be satisfied always. So, we are looking
for the nonzero vector here which is called
the eigenvector and this is called the eigenvalue
ok.
So, now the vector x is the eigenvector associated
with this eigenvalue lambda and the two important
points now, the geometrically which we have
already seen with the help of earlier example
that an eigenvector of a matrix A is a nonzero
vector, x in this R n such that the vectors
here x and this Ax are parallel. This is what
we have seen in previous example that this
vector v and the vector Av they were just
the parallel, the length was different. So,
here also the geometrical meaning of this
eigenvalues eigenvector in general is that
of this vector this x and this Ax both are
parallel and their magnitude will change and
therefore, we have this eigenvalue lambda.
Algebraically, an eigenvector x is a non-trivial
solution because we are looking for the eigenvector
x which is nonzero. So, meaning this non-trivial
solution because x is equal to 0 will always
satisfy this equation. So, we are not interested
in the 0 solution, our interested in nonzero
solution; meaning the non-trivial solution
of this equation Ax is equal to lambda x.
Or, this eigenvector x here is a nonzero vector
in the null space of this A minus lambda I,
because this equation which we have a is equal
to l Ax is equal to lambda x. So, this Ax
is equal to lambda x what we can do we can
bring this lambda x term to the left hand
side. So, we have A minus this lambda with
the identity matrix because, we need to subtract
from this A. Then we have to also introduce
this identity matrix and then x is equal to
0.
So, basically what we are looking for, we
are looking for the non-trivial solution.
This x of this equation A minus lambda I or
rather the system of linear equations A minus
lambda x is equal to 0 and this is exactly
the definition of the null space of this A
minus lambda I. So, here the matrix is A minus
lambda I and if we look for the null space
of this A minus lambda I.
So, this x vector which we are looking for
is in the null space of this matrix A minus
lambda I. And, since it is a nonzero I mean
the null space also has a now has a 0 vector,
but we are looking for the nonzero vector
in the null space of this. So, x is a nonzero
vector in the null space of this A minus lambda
I. Now, the natural question is how to compute
this eigenvector and how to compute the eigenvalues
associated with this with the matrix of order
n.
So, how to find the eigenvalues and eigenvectors
we will discuss now; so, consider this equation
A minus lambda I x is equal to 0, that is
Ax is equal to this lambda x equation written
in this form. And, there are two unknowns
here, the unknowns are the lambda we need
to compute lambda and also we need to compute
x. There is a one equation here A minus lambda
I x is equal to 0 or it is a system of linear
equation and we have these two unknowns the
lambda and x. So, how to compute these two
unknowns so, that those unknown satisfy this
equation A minus lambda I x is equal to 0?
The lambda is a scalar and this x is a vector
whose components will be exactly equal to
this n, if the matrix is n cross n matrix.
So, what is other information we have that
we are looking for this non-trivial solution
x.
So, this equation A minus lambda I x has a
non-trivial solution which we have already
studied in previous lecture. If and only if
this satisfy the equation, which equation
that the determinant of this A minus lambda
I; if this determinant is 0 then we will have
a non-trivial solution. If the determinant
is not equal to 0 then there will be a unique
solution and that will be the trivial solution
meaning this x will be 0.
But, we are looking for a non-trivial solution
of this equation A minus lambda I x is equal
to 0 and in that case we have this condition
that this determinant of this A minus lambda
I matrix this must be 0. So, we got another
condition which is leading us to at least
get now something out of this condition determinant
of A minus lambda I is equal to 0. So, when
we expand this determinant because, this lambda
is the unknown now A is a given matrix and
I is the identity matrix.
So, here this lambda is unknown. So, when
we expand this determinant, determinant is
nothing, but this polynomial equation there
c 0 lambda power n plus c 1 lambda power n
minus 1 and so on c n, these are the coefficients
and they will be naturally the given when
this A is given. So, we have this polynomial
equation and then if we solve this equation
we will get at most these n roots of this
equation; that means, the n values of the
lambdas. They may be distinct and they may
not be distinct, they may be real, they may
not be real so whatever. So, the solution
of this equation which is called the characteristic
equation. So, this equation is called characteristic
equation of the matrix A and after solving
this equation we can get the possible lambdas.
Once we have the lambda here then we have
this equation A minus lambda I x is equal
to 0 and for each lambda we can find the solution
of this A minus lambda I x is equal to 0.
And, note that our lambda which we will get
with this condition that the determinant is
0. So, naturally we will get non-trivial solution
of this A minus lambda I x is equal to 0 because,
the trivial solution will come when this determinant
is not equal to 0. But, we have we will get
our lambda such that this determinant A minus
lambda I will become 0. And therefore, automatically
for these lambdas we will get the non-trivial
solution of these A minus lambda I x is equal
to 0.
So, here the roots of this characteristic
equation are exactly called the eigenvalues
because this lambda is the eigenvalue. So,
once we solve this characteristic equation
we will get all the eigenvalues and the eigenvectors
of A can be determined by this solving the
homogenous system of this linear equation;
that means, this A minus this lambda I x is
equal to 0. So, we need to solve again the
system of linear equations. So, from the beginning
of this lectures in linear algebra I am emphasizing
again and again on this system of linear equations
because at each and every step finally, we
are solving the system of linear equations.
So, it is very important now and here for
each value of this lambda we need to solve
the system of equations.
So, if we have 3 distinct lambdas for example
so, for each lambda. So, for 3 times we have
to solve the system of linear equations and
they will be different equations because we
have the different lambda. So, this matrix
is going to be different and we will get different
eigenvectors. So, we will be talking more
on this now and this null space of this A
minus lambda I. So, the null space means these
x I and which includes the 0 also because
the null space will also include the 0. So,
the null space here called the eigenspace
of A corresponding to eigenvalue lambda.
So, what is in the null space? In the null
space carries all the eigenvectors plus the
0 vector because the 0 is not the eigenvector.
So, here in the null space which is also called
the eigenspace, it contains all eigenvectors
including 0 vector because 0 is naturally
the solution of this A minus lambda I x is
equal to 0 or 0 will be there in the null
space. But, 0 is not the eigenvector because
the eigenvector we define as the nonzero,
nonzero x the non-trivial solution of this
equation. So, another terminology here which
we may use later that is eigenspace; so, eigenspace
is nothing, but the set of all these eigenvectors
including the 0 vector ok.
So, let us go through the problem here. The
problem number 1 it is a find the eigenvalues
and the eigenvectors of this matrix A is equal
to 2 1 4 n minus 2; it is a very simple example
we start with the evaluation of these eigenvalues
and eigenvectors. So, here first we have to
write down the characteristic equation and
we need to solve the characteristic equation
always to find the eigenvalues. And, then
for each characteristic value or each eigenvalue
we have to solve that system of equation that
now we will get in that way the eigenvectors.
So, here the characteristic equation is the
determinant of this A minus lambda I.
So, A minus lambda I; that means so, the eigenvalues
are nothing, but the this solution of this
characteristic equation. So, we have this
A minus lambda I. So, A here 1 2 1 and this
4 and minus 1 and then we have this lambda
I that will be the determinant at the end.
So, lambda I means the lambda 0 0 and the
lambda that is the product of lambda and this
identity matrix and this we want to solve
know the determinant here. So, what is the
matrix? The matrix here is 2 minus lambda
and then we have here 1 and here we have 4
and then minus 1 and the minus lambda, that
is the determinant here which we can directly
always we can read write down for this given
matrix A.
So, the A was given this 2 1 and 4 minus 1.
So, how to write the characteristic equation?
Just the determinant of this matrix subtracting
lambda from the diagonal; so, 1 minus lambda
I so, lambda from the diagonal here how we
will get this 2 minus lambda and minus 1 minus
lambda is equal to 0 and now the determinant
value here this product. So, which will be
2 minus lambda and multiplied by this minus
1 minus lambda and this is minus 4 here is
equal to 0. So, we can get this product that
will give minus 2 and then here minus 2 lambda
plus lambda and plus lambda square and then
we have minus 4 is equal to 0.
So, we get this minus 6 here they send this
then we have this lambda square and then we
have minus lambda. So, this is the characteristic
equation which we can factorize here easily
and that is my lambda minus 3 and this lambda
plus 2 is equal to 0. So, that is the characteristic
equation from there we can get the roots of
the equation. So, that is what given here
the lambda minus 1 and lambda plus 2 is equal
to 0. So, that is our characteristic equation
here.
And its root that will be the eigenvalue;
so, eigen values are the 1 eigenvalue here
which we are calling lambda 1 that is 3 because,
this is the solution of this equation. And,
the another eigenvalue will be lambda 2 which
is minus 2 and the eigenvector now corresponding
to each here. So, first let us take this lambda
1 is equal to 3. So, while taking this we
have to now form the system of equation A
minus lambda I and times this x here so, that
will be the system of equations. So, we have
to subtract this 3 from the diagonal entries
and that will be our matrix here.
Meaning so, if we do so, so here A minus 3
I. So, A was 2 here so, if we subtract 3 from
the diagonal entries we will get minus 1,
here 1 and here 4 and this is minus 1 and
then minus 1 so, minus 3. So, this will be
minus 4 and then we have these x 1 x 2 4 x,
it has 2 component and the right hand side
is the 0 vector this one. So, we have this
system of equation which we need to solve
to get the eigenvector corresponding to this
lambda 1 is equal to 3 and this is simple
because, our this matrix here which I am again
calling this A. So this is the matrix which
we can easily get to the reduced echelon form.
So, row reduced echelon form for this matrix
will be minus 1 and this 1 and here we can
multiply by 4 and add it to this equation
so that will give 0 0 and that is the row
reduced echelon form of this matrix from where
we can easily identify the solution. So, here
this is our pivot element the first one minus
1 and here we have the 0 rows. So, naturally
we will get a non-trivial solution because,
the lambdas were obtained with that condition
that we will a non-trivial solution; meaning
always you will get for solving such system
some free variables.
So, here for instance this x 2 we can call
as free variable. So, this x 2 is a free variable
we can choose a value whatever value we like.
So, if we choose x 2 1 then this equation
will give us; so, minus x 1 plus x 2 is equal
to 0 that is the equation here and if we choose
x 2 is equal to 1 so, x 1 will be also 1.
So, one solution of this and there are infinitely
many possibilities of the solutions. So, here
the one possibility of the solution is that
x 1 is 1 and x 2 is also 1. So, that is a
solution what we get out of this, that is
a one solution any multiple of this so, we
can have 2 2 or any number we can multiply
to this one that will be the solution of this
equation meaning eigenvector.
So, eigenvector is never unique here corresponding
to this lambda 1 is equal to 3 we got this
vector 1 1 a 1 vector that is also need to
be mentioned because, the 2 2 will be also
the solution and that will be also the eigenvector.
So, any multiple of this will be the eigenvector
because that is a solution of this and this
is the kind of generator of the solution or
the basis of the null space of 4 of this A
minus 3 I matrix. So, that is the basis here
for this null space of this A minus 3 I and
that is the vector here x which we call the
eigenvector. So, same similar steps we will
have to repeat now for lambda 2 is equal to
minus 2 and maybe I can skip that. So, here
we have this a plus 2 I now.
So, this 2 will be added now to the diagonal
entries. So, we will have A plus 2 I will
be this 4 1 and 4 minus 1 as the matrix and
from there we can get as again 1 vector here
1 minus 4, but any multiple of this will be
also the eigenvector. So, by solving this
equation A plus 2 I is equal x is equal to
0 we got the another vector here which is
1 and minus 4. What is also interesting and
we will note it down later on that there were
two different eigenvalues here 3 and minus
2 and, their eigenvalues 1 1 or any multiple
of this 1 and 1 4. What we can also check
and we can indeed easily see here in case
of these two vectors that these two are linearly
independent vectors. We cannot get like lambda
1 times this 1 1 and then lambda 2 times this
1 minus 4.
If you want to set this linear combination
to 0 the only solution will be that lambda
1 is 0 and lambda 2 is 0, there is no other
possibility here. So, these vectors 1 1 and
1 minus 4 they are linearly independent vectors.
And, later on in the theoretical result also
we will see that whenever we have two different
or n different eigen values then corresponding
the eigenvectors will be linearly independent,
that we can theoretically prove. We will prove
actually in future lectures. So, here these
are the 2 eigenvectors which we have easily
evaluated corresponding to each of the eigenvalues.
So, another example here find the eigenvalues
and these eigenvectors of A which is given
here again a very simple matrix we have taken
just for demonstration 1 1 and 0 minus 1.
So, if we write down the characteristic equation.
So, that will be A minus this lambda I is
equal to 0 so; that means, we will subtract
here from the diagonal entries lambda. So,
that will be a 1 minus lambda 1 0 and this
1 minus lambda is equal to 0.
So, which the product here 1 minus lambda
square is the characteristic equation and
their roots will be then lambda 1. So, here
the characteristic equation is coming 1 minus
lambda whole square is equal to 0. So, there
are two roots of the characteristic equation
and these two roots are the lambda 1 is equal
to 1 and lambda 1 2 also 1. So, it is a repeated
root the case of the repeated root we have
only this one eigenvalue which is repeated
2 times and now corresponding to this eigenvalue
we need to compute the eigenvector. So, the
eigenvector corresponding to corresponding
to these values here 1 1 we can again form
the system of linear equation that is A minus
this 1 lambda times 1.
So, lambda is 1 here so, A minus 1 x is equal
to 0. So, this system of equation we have
to solve now and what is the system now A
minus 1. So, here 1 will be subtracted from
the diagonal entries and what we will get;
so, let me just first see here. So, our matrix
will be now the 0 and 1 and 0 and then here
also this 0 that is our system of equation
x 1 x 2 and is equal to this 0 0 the right
hand side vector. So, this is the row reduced
echelon form already and we have here this
pivot element; that means, this x 2 is not
a free variable, but here this x 1 because
the first column does not have a pivot element.
So, this is what we call the free variable
and this free variable we can assign any value
we like. So, this x 1 we are assigning some
alpha for instance and this x 2 equation that
is already given from the first equation that
x 2 is 0. So, we do not have even dependency
on this alpha, the x 2 is always 0 whatever
alpha we take that is the freedom we have
now. So, our solution of this system of equation
is x 1 and x 2 this alpha and this 1 0. So,
any alpha we can we can choose of course,
here and that is the solution. So, this 1
0 is the is the generator here for the solution
and now that is what we have this 1 0 is one
of the eigenvectors corresponding to 1 and
any multiple of this will be also the eigenvector.
So, here we have seen there were two different
eigenvalues, but we get only one eigenvector.
So, that is also possible which can be seen
here with the help of this simple example;
more on this we will be talking about later
that what are the possibilities corresponding
to 1 this eigenvalues how many eigenvectors
are possible and so on.
So, there is a Cayley-Hamilton theorem which
says that every square matrix satisfy its
own characteristic equation, that is another
result which directly coming from the characteristic
equation that every square matrix satisfies
its own characteristic equation. So, we know
what is the characteristic equation that is
the determinant of this A minus lambda I;
meaning this such polynomial equation is the
characteristic equation. And, this result
says which we will not go through the proof
that this u this matrix itself will satisfy
this characteristic equation; that means,
if instead of the lambda if we replace this
by A.
So, here A power n minus 1 and then this and
so on up to c n this will be in that case
c n into I, I will be the identity matrix
of the same size. So, that this Cayley-Hamilton
theorem says that every square matrix also
satisfies its characteristic equation and
that means, that A will also satisfy this
equation. So, it is just the lambda is replaced
by this A here and with the c n to make it
consistent because, now we are working with
the matrices. So, there should be a matrix
here of the same order.
So, that is the result of the Cayley-Hamilton
theorem which we can verify for instance for
this very simple example which have taken
11 minus 6 i 4 i and 1 and we will verify
this characteristic polynomial by this or
de Cayley’s-Hamilton theorem. So, the characteristic
polynomial characteristic equation we have
to write corresponding to this A which will
be just by subtracting this lambda from the
diagonal entries and now we can write down
in terms of this I mean this determinant here.
So, that we will get this equation lambda
square and minus 12 lambda minus 13 that is
just the product of 11 minus lambda 2 1 minus
lambda and then the minus minus plus here
24 i square.
So, again minus 24 so, that we can simplify
we will get this equation lambda square minus
12 lambda minus 13. And, now what we will
see the Cayley-Hamilton theorem says that
this A square minus 12 A minus 13 I this should
be just 0. So, this should satisfy the characteristic
equation and if we compute this A square that
is coming to be here. So, the product of this
A with the matrix A that will be coming this
one here minus this 12 times the A. So, 12
is multiplied to each of the entry of the
A and then we have minus 13 I here.
So, when we when we determine this one so,
here this minus so, these two will be added
and that this will be subtracted; we are getting
actually 0 matrix. And, that is what the characteristic
equation and this Cayley-Hamilton theorem
says that every square matrix satisfy its
characteristic equation. So, we have just
replaced here lambda by this A and then we
have seen that this right side instead of
the 0 we got the 0 matrix.
So, that is what the Cayley-Hamilton theorem
is; another use of this Cayley-Hamilton theorem
we can quickly look from this example we can
use this Cayley-Hamilton theorem to prove
this A inverse. So, again very simple example
we have started with this A is equal to 2
4 and 3 5. So, if we write down its characteristic
equation. So, again this lambda is subtracted
from the diagonal and we can simplify this
determinant. So, we will get this lambda square
minus 7 lambda minus 2 is equal to 0 that
will be the characteristic equation.
And, by Cayley-Hamilton theorem we know that
this A square minus 7 I minus 2 I will be
equal to 0 and which we can write we can take
this common here in these first two terms
this A. So, A into A minus the 7 the identity
matrix will be introduced and is equal to
this 2 I. And, now what we can do here we
can multiply by A inverse again a point to
be noted that we can multiply by the inverse
when the A inverse exists.
So, up to now this is the Cayley-Hamilton
theorem says that every square matrix satisfy
its own characteristic equation, but if you
are now multiplying by A inverse here; so,
that is only possible when A inverse exists.
So, we should not do such calculations when
A inverse does not exist. So, we should verify
first whether A inverse exists or not in this
case naturally it exists. So, here we multiply
them by A inverse and what we will get the
right hand side will be A inverse and this
2 we can divide here. So, we will get 1 by
2 A minus 7 I that should be the A inverse.
So, very easily we got this A inverse here.
So, we need to do just a simple calculations
to get this A inverse A minus the 7 times
I. So, that is the value of this A inverse
using the Cayley-Hamilton theorem.
So, coming to the conclusion what we have
done here, we have studied, we have introduced
the eigenvalues and eigenvector and basically
this equation was very important this Ax is
equal to lambda x. This lambda is the eigenvalue
and the corresponding eigenvector will be
given by x. And, we have also studied this
Cayley-Hamilton theorem which says that every
square matrix satisfy its own characteristic
equation. So, here the right hand side this
is a 0 matrix so, the same order having all
the entries 0.
And, these are the references used for this
for preparing these lectures.
Thank you very much for your attention.
