So, welcome back and today again we will continue
our discussion on these eigenvalues and eigenvectors.
This is lecture number 50 and we will mainly
focus on iagonalization and also it is applications
for solving system of linear equations, also
for getting the power of the matrices etcetera.
So, what is the diagonalization of the matrix?
We will discuss here now.
So, A square matrix A is said to be diagonalizable
if there exists an invertible matrix P. If
there exists an invertible matrix P such that
this P inverse AP is a diagonal matrix or
in other words, because this concept we have
already introduced the similarity of the matrices.
So, in other words if A is similar to a diagonal
matrix, because if the point is here A is
called diagonalizable. If we can write down
this A as this P inverse the P inverse AP
is a diagonal matrix then we call that this
A is diagonalizable. So, the meaning this
A is similar to the diagonal matrix here;
AP inverse AP and this P is invertible matrix
which we have to find, so that this P inverse
AP becomes a diagonal matrix.
So, let A be an n cross n matrix and that
is a nice result that A is always diagonalizable;
that means, we can find such A P such that
this P inverse AP is a diagonal matrix. So,
the result is here A is diagonalizable, if
A has n linearly independent eigenvector.
So, that is a nice result here, nothing to
do with this eigenvalues. Actually we have
to look for the eigenvector. So, if we get
n linearly independent eigenvectors then A
can be diagonalized. And if we cannot get
these n linearly independent eigenvectors
then the A is not diagonalizable. So, that
is the main result of this lecture.
Again we will not go through the formal proof
here, but we will see with the help of many
examples, how this is working. And another
subsequence of this result we can we can have
here if n is an n cross n matrix here A and
it has n distinct eigenvalues, then also A
is diagonalizable and then reason is clear,
because if we have n distinct eigenvalues,
then we will also get n linearly independent
eigenvectors; that is a result we have already
seen in previous lecture that corresponding
to distinct eigenvalues we have a linearly
independent eigenvectors. So, eventually this
second result here is again the same as this
previous one; that is diagonalizable, if and
only if it has n linearly independent vectors.
So, just a note here that this matrix P which
diagonalizes A is called the model matrix
of A and whose columns, I mean the point is
how to find this A matrix P. So, here this
model matrix P has the columns that are nothing,
but the eigenvectors corresponding to these
different eigenvalues. So, if you have n linearly
independent eigenvectors, we will just place
them in this matrix P as the columns, and
this is our matrix this P. And when we check
this P inverse AP that will be nothing, but
the diagonal matrix and entries of these diagonal
matrix will be just the eigenvalues, corresponding
to these eigenvectors we have placed in the
sequence as columns of this matrix P.
So now here let us consider this example with
A 5 4 and 1 2. So, here we will not spend
much of our time for computing eigenvalues
and eigenvectors, because that we have already
seen in previous lectures. So, we can compute
the eigenvalues for this example and then
it comes to be 1 and 6, because the idea is
simple that this 5 minus lambda and then we
have 4 and 1 2 minus lambda.
So, this determinant we have to solve which
we can make this product here, the 10 and
then we will have here minus 2 and minus 5.
So, minus 7 lambda and plus this lambda square
and minus 4 is equal to 0. So, this is lambda
square minus this 7 lambda. And then we have
here 6 is equal to 0 and that can factorize
to this minus 6 minus 1. So, lambda minus
6 and lambda minus 1 is equal to 0. So, here
we get these eigenvalues as 1 and 6.
So, having these eigenvalues now corresponding
to each, we have to find the eigenvector.
And here we have indeed these distinct eigenvalues.
So, we will get two linearly independent eigenvectors
and then we can diagonalize this matrix. So,
here the eigenvectors corresponding to 1 will
be coming as this minus 1 1. So, that is corresponding
to this one, and corresponding to 6 we will
get here 4 1. And now once we have the eigenvectors
we can formulate this model matrix which we
call P. So, the P will be we are placing these
matrices are these vectors the eigenvectors
as columns of this P.
So, here minus 1 1; that is the first column,
and then this 4 1 that is the second column.
So, our model matrix is ready now and we can
verify that how this P inverse AP A P look
like. So, here we have to get this P inverse
also, that is the inverse. So, for 2 by 2
matrix it is simple, so we have to divide
here by this determinant and then we need
to change the sign and determined will be
again with minus sign.
So, finally, we will get this as the as the
P inverse of this matrix P which we can also
verify by multiplying these two and we are
getting the identity matrix. So, here the
P inverse and if we compute this P inverse
AP, so that is coming to be 1 6 in the diagonal
and it is a diagonal matrix and this is exactly
the point here. So, we have kept in our model
matrix this, these vector minus 1 as the first
column, and that was corresponding to the
eigenvalue 1 and that is the reason here this
first eigenvalue is coming and in the second
case we have kept this second column which
was corresponding to the 6 here and therefore,
the second element in the diagonal is 6 here.
So, this order if we change for instance the
order here.
So, if we change it to P like 4 1 and minus
1 1. If we change, if we take this P, if we
take this model matrix then here P inverse
AP when we compute, this will be 6 0 and 0
1, because here this was corresponding to
this 6 and then this is corresponding to this
number 1 here. So, accordingly that order
will change.
So, the order we place here for the eigenvectors
corresponding order will be followed in the
diagonal entries as the eigenvalues, so that
is important. Second point here which also
needs to be mention that this is not the unique
eigenvector for instance; so, we can multiply
by any number to this minus 1 1 that will
be also the eigenvector. Again here also this
4 1 we can multiply by any scalar that will
also be the eigenvector, because eigenvectors
are not unique and by doing so, also there
is no problem we can keep any vector here
not only minus 1 and 1, we can also place
for example, minus 2 and 2 here in the first
column.
And in the second column we can place for
instance we multiplied by 2 here, so 8 and
2. So, we can place 8 and 2 in the second
column. So, does not matter that will be taken
care by this P inverse and still this product
will give us the same diagonal matrix 1 0
0 6. So, here it is material that whether
we multiply here to these eigenvectors by
some scalars; it does not matter with this
P inverse AP will lead to the same eigen,
same diagonal matrix whose entries will be
1 and 6. The only thing matter again it s
the order here we place these eigenvectors.
So, the order we keep here placing as these
columns of these eigenvectors in the same
order these eigenvalues will appear.
So, here another example of this 3 by 3 matrix
6 minus 2 2 and then we have minus 2 3 3 and
minus 1 2 minus 1 3. So, for this matrix also
if you want to check whether it can be diagonalized
or not and what will be the diagonal matrix?
What will be the model matrix? So, for that
we need to compute the eigenvalues. So, the
eigenvalues for this matrix will be coming
2 2 and 8.
So, again now for each eigenvalue we need
to compute the eigenvector to form this model
matrix P and the corresponding to this eigenvalues
this 2 2 the repeated 1. So, here the algebraic
multiplicity of this 2 is 2 and now we compute
the eigenvector corresponding to this. So,
here this 2 minus 2 and minus 2, so that will
be the matrix there which we want to solve
as the system of linear equations. So, by
doing so what we are actually getting here
we are getting 3 linearly independent eigenvector
meaning this geometric multiplicity of 2 is
2 and also it is; as the algebraic multiplicity
is 2, also the geometric multiplicity in this
particular case is coming to be 2.
So, this is corresponding to this 2 this is
also corresponding to 2 and here we have this
corresponding to this 8. So, we have 3 linearly
independent linearly independent eigenvector
and that is the reason now we can actually
diagonalize this matrix because we need 3
matrices. Remember it is easy to remember
here the model P here the model matrix P will
be of the same order as A. So, we need this
3 columns to fill the matrix P. So, if we
have 3 linearly independent vectors we can
form this P otherwise for example, corresponding
to 2 if it happens that we have only 1 eigenvector
then we cannot form this P in other words
the matrix A is not diagonalizable in that
case.
So, here the matrix A is diagonalizable because
we are getting 3 linearly independent eigenvector
and the model matrix P here we will place
this 1 to 0 minus 0 2. First two columns and
the corresponding to 8; we have this 2 minus
1 as a third column. So, this is corresponding
to 2. The first column this is also corresponding
to 2 and this corresponds to 1 the third column.
So, our order will remain exactly this one
and this will become the diagonal entries
of the matrix P inverse AP. So, if you compute
the P inverse AP now.
So, we need to get this P inverse and then
this product we have to make and then we will
get this 2 2 and exactly the order we have
placed here these eigenvectors. So, we are
getting these diagonal entries absolutely
the same here 2 2 8. So, that is the diagonal
matrix here which is similar to the matrix
A and later on we will observe several good
properties about this matrix because they
share many common properties these similar
matrices and some of the applications very
important applications one we can once we
can diagonalize the matrix we can we can use
them in many applications.
So, that will be the also topic of discussion
of this lecture.
So, the last example here we will take another
one where we do see this is the lower triangular
matrix with entries 2 2 3 in the diagonals
and then we have this 4 in the off diagonal
rest everything is 0. So, in this case if
we compute the eigenvalues we know for the
triangular matrices. So, this is 2 2 3. So,
the eigenvalues will be 2 2 and 3 and we have
to compute again the eigenvectors corresponding
to the 2 and also corresponding to this 3.
What happens in this case that here this algebraic
multiplicity of 2 is 2 and it comes to be
that the geometric multiplicity of this 2
is 1 and that is the point where actually
we cannot diagnolize the system because geometric
multiplicity is not equal to the algebraic
multiplicity for this eigenvalue 2 then we
will get less number of eigenvectors and we
cannot form this model matrix P.
So, here the corresponding to these 2 eigenvalues
the repeated eigenvalue we are getting only
1 eigenvector and the reason is clear because
if we formulate this equation A minus lambda
I x is equal to 0. So, what will happen here?
We get this 0 0 0 and then we have here 4
this again 0 0 and 0 0 1. This is the situation
of this system of equation for the eigenvector
here and the right hand side 0. So, for this
system of equation what do observer what do
we observe here that we have the 2 pivots
element here 4 and also this 1. These are
the pivot elements and the free variable we
have only one that is here x 2.
So, x 2 is free variable free variable; that
means, we can choose this x 2 whatever we
like. So, let us take this alpha and then
directly from these equations we have observe
that the x 1 is 0 from this second equation
and from this third equation we observe that
x 3 is equal to 0. So, the eigenvectors x
1, x, 2, x 3 in this case corresponding to
this repeated eigenvalue is coming to be 0
1 0 and any multiple of this 0 1 0.
So, that is here we have taken just alpha
1. So, this is one of the eigenvectors here
and then corresponding to 3 also we can compute
the eigenvector and that is naturally it will
come 1 only. So, we have 2 minus 1 and 1.
So, with these 2 vectors because we need 3
vectors to fill the positions of this model
matrix P. So, we cannot do in this case because
we are getting 2 linearly independent eigenvectors
and therefore, this matrix A is not diagonalizable.
So, the given matrix is not diagonalizable.
So, what we have seen here that every matrix
we cannot diagonalize. We can diagonalize
only those matrices when the eigen vectors
the set of this eigen vectors is full means
if it is a n by n matrix then if we get n
linearly independent Eigen vectors then we
can diagonalize the matrix, otherwise we cannot
diagonalize the matrix.
Now, coming to the applications of the diagonalization,
the first application we will consider that
we can easily compute the power of the matrices
once we can diagonalize the matrix. So, why
so? What is the connection here or to the
to the eigenvalues, eigenvectors that we will
see now. So, this P inverse AP what we have
seen that A can be diagonalized then we have
this relation P inverse AP is equal to D or
we can rewrite it that A is equal to so we
multiply by P here first there will be PD
and then the right side we will multiply by
P inverse.
So, we will get out of this relation here
A is equal to PD and P inverse. So, having
this relation then if you want to multiply
or we want to get this A power 2 or A square
in that case. So, we need to multiply this
PDP inverse with the PDP inverse and then
with this associativity property of this product
we will realize here that this P inverse,
P is there which we can put as the identity
matrix and then we will get here P D and D
P inverse meaning this P and the power of
this diagonal matrix. So, here the A square
the power of this matrix is 2 power of this
matrix; it is coming to be that this power
is exactly translated to the power of this
diagonal matrix. What is the use here that
this diagonal matrix we can easily get this
power here because this power will directly
go to this diagonal entry. So, we do not have
to actually multiply these matrices D here
for this power, but only the diagonal entries
will be squared and that is a reason here.
So, the A square is very simple now the PD
square P inverse. So, we have to only do these
multiplications. Naturally when we have a
high power here and not just for the two because
in any case now we have to do this multiplication
PD square and also this with the P inverse.
So, here naturally the work is more if we
just want to find out this power 2, but in
case for example, we want to power to get
the power 1000 then definitely this will be
very very useful because D power higher power
would be easier to compute. So, why this A
square is coming D square? We can continue
this idea for example, A 3 also the same similar
structure will happen.
So, this PD square P inverse that is for A
square and then again multiplied by A and
this P inverse P will again become the identity
matrix and we will have PDQ P inverse. So,
what is the point here that we can see out
of these calculations that A power n will
be also just D power n here and this PD power
and P inverse. So, this will continue this
power here on translating to this matrix T.
So, in general also we can prove that this
A power n is nothing, but the P D power and
n P inverse. So, we can use as I said before
when we want to compute this very high power
of this matrix a large number here and then
this is very very useful because D power n
the computation here is very simple because
if we take 2 diagonal matrix for example,
this here and we want to multiply with the
same.
So, what will happen now? So, if you multiply
this a square will come and then this product
will be 0 here. Also this will be 0 and b
square will come. So, what happens when we
do the product of the diagonal matrix just
simply we will this power; this power will
go to the diagonal entries. So, we do not
have to do this multiplication as the matrix
multiplication just simply when we have the
power n. So, for instance we have this a 0
0 b and we want to get this power n there.
So, this will be nothing, but the a power
n zero and b power n. So, that is the point
here. So, having this relation that A power
n is nothing, but the D power n here. So,
that is very simple to compute and then finally,
we need to make this product with the P and
the P inverse. So, that is the only computation
load here for this matrix multiplication,
but for D power n only that linear relations
here. So, these diagonal entries will be powered
and nothing else.
So, let us just go through one example which
says this find A power 5 for A is equal to
1 4 and half 0. So, but to do so, we have
to compute the eigenvalues and also the eigenvectors,
because we need that P and we also need that
diagonal matrix. So, in this case the eigenvalues
are coming to be minus 1 and 2 and then we
compute the eigenvectors corresponding to
minus 1 it is 2 1 and corresponding to 2 it
is coming as 4 1. So, having this we can now
form this P the model matrix P. So, placing
these eigenvectors here as the columns.
So, the first column we have 2 minus 1 and
the second column we have this 4 1. So, having
this model matrix now we can compute this
P inverse again easily and then this A power
P as per our discussion in the earlier slide.
So, we have this P and the D power P and P
inverse. So, we have to get this power of
the diagonal matrix and then we have to multiply
by this P and P inverse. So, looking at this
one, so we have this P inverse and then this
D; D was; so, what will be the D? We have
the eigenvalues minus 1 and 1.
So, this D will become simply we have to place
these eigenvalues in the order we have placed
these eigenvectors in P. So, this was the
corresponding to 2 minus 1, the second was
corresponding to 2 2. So, therefore, this
order will be maintained here in the eigenvalue.
So, that is the matrix this D here. So, the
D power 5 is just minus 1 power 5 and this
2 power 5 which is here.
So, we can actually compute a very high power.
Also the computational load will remain the
same. Only thing we have to we have to just
do this power here of the diagonal entries.
So, and then later on we have to in any case
just multiply by this P and the P inverse.
So, it is easy now to find having this relation
any power of the matrix A. So, here the A
power 5 when we do this multiplication that
is coming this 21 44 and this 5.5 and this
10. So, that is A power 5, but it is usually
used when we really want to have we want to
compute a high power of this matrix A then
this is computationally very very efficient
as compared to doing the product of matrices
A.
So, here was the one of the applications where
we use this eigenvalues, eigenvectors or in
particular this idea of the diagonalization
of the matrix.
Now, coming to the next application which
is the solution of the system of linear differential
equations though the differential equations
will be the topic of the next few lectures,
but here just to introduce the idea of this
or the application of this diagonalization.
We will be doing very simple example also.
So, here we consider the linear differential
equations. So, here it is a system, system
means we have a more than one differential
equations and their variables are coupled.
So, here we have this; the left hand side
for example, this is the derivative term.
So, meaning that we have like this dx 1 dt
and suppose we have two equations dx 2 over
dt and then right hand side some matrix is
given here. So, a 1 1, a 1 2, a 2 1, a 2 2
and then the variable here X 1 and X 2. So,
we have these two unknowns for example, in
the system X 1 and X 2 and this is the system
because the first equation of this system
dx 1 over dt is having this X 1 and X 2. So,
a 1 1 X 1 plus a 1 2 X 2 and the second equation
here is dx 2 over dt a 1 1 a 2 1 X 1 and plus
a 2 2 X 2. So, we have basically these two
equations. These two are the differential
equations we have the ordinary derivative.
So, these are the system of ordinary differential
equations and these are coupled because this
X 1 and X 2 is percent in this equation 1
as well as in the equation 2. So, these are
the coupled equations and they are not very
easy to solve, but with the idea of this diagonalization
it becomes very very simple as we will observe
now here. So, what we have to assume? We have
to assume that this matrix A is diagonalizable;
this coefficient matrix here is diagonalizable.
So, once we know that.
So, we know this relation that D is equal
to P inverse AP. So, this D then this system
here which was the derivative term of this
X is equal to A is replaced now by this PDP
inverse. So, PDP inverse and the X the right
hand side and what we do? We multiply this
to P inverse now. This equation to where P
inverse is derivative the vector of the derivative
terms then we have DP inverse and this X term.
Now this is just the coefficient it is just
the matrix here, which we have the model matrix.
So, these are just the constant they increase
your P, they are the constant. So, we can
actually rewrite this P inverse and the derivative
terms here are coming in this vector. So,
we can simply take the derivative of this
P inverse X because these are the constant
terms. So, this will not matter for the derivative.
So, we have collectively taken this as P inverse
X and the derivative right hand side PD and
this P inverse this X and now what we do we
just substitute a new variable here we take
for this P inverse X.
So, substituting this P inverse X is equal
to Y a new variable. We have named the new
variable when we multiply this P inverse to
X. We are setting this new variable Y and
our differential our system here which was
this X dot is equal to AX t, which is now
reduced to this Y dot is equal to DY t. So,
what is the benefit now; having from this
system to this system. Now do you notice that
this t is the diagonal entries here now in
the D and they are the eigenvalues here are
placed in this t.
So, the benefit here that our system the reduced
system is that here we have the derivatives
from the, here we have this diagonal matrix
and then we have the y the unknown. Now if
we just look at this multiplication what we
are getting? We are getting these n equations
and they are actually decoupled equations
now because once we multiply this with the
diagonal entries what we are getting lambda
1 y 1, the second entry here lambda 2 y 2
and lambda n y n and here also we have these;
the derivatives of y 1, y 2, y 3, y n.
So, we got here the i equation I mean n equations,
but they are decouple now because the for
instance the first equation has only y 1,
the second equation has only y 2, the third
equation is having only y 3 and the single
equation this dy over dt is equal to lambda
y type equations we know how to solve because
all these equations are of this type dy over
dt is equal to yt and we know that the solution
here that this is y as some constant exponential
t. So, that is the solution of this equation
y is equal to c exponential time t and here
we have all these equations which are no more
coupled equation.
So, by this idea of the diagonalization we
got this from the coupled system of equations
just these uncouple system of equations, which
are easy to solve now. So, we will solve these
each question here the solution will be y
i t is equal to C i and e power lambda i t.
So, the C isare this constant of integration.
So, what do we get? We have the P inverse
X t that was the our substitution which we
have made for Y and now we got the vector
Y. So, from there we can indeed get this X
again back because our main variable in the
given system was X or we can write down this
in this expanded form.
So, this X was having these n components x
1, x 2, x 3, x n. Here this is P the vector
P; the vector P here is having the columns
as the eigenvectors right. So, here the d
was the diagonal matrix whose entries were
lambda 1, lambda 2, lambda n and here we will
get the corresponding eigenvector. So, v 1,
v 2 and this v 3 and so on and then we will
get this v n. So, these are the corresponding
eigenvectors of these eigenvalues lambdas.
So, this was the model matrix here.
So, then we can do this product as well. So,
this product with the Y t. So, here we have
basically y 1, y 2, and y n t whose value
also we know that C 1 e power lambda i t.
So, this matrix vector product we can take
as the first column multiplied by this first
element, the second column multiplied by the
second element and so on. This is what we
have written here for Y we have substituted
this C 1 e power lambda 1 t, here C 2 e power
lambda 2 t, C n e power lambda n t.
So, what is the final remark here that we
need to compute the eigenvalues of the given
matrix A and also the eigenvectors and then
we can write down finally, the solution directly
that the constant terms the first eigenvector
a corresponding to this lambda 1. So, e power
lambda 1 t, this eigenvector corresponding
to lambda 2, eigenvector corresponding to
lambda n. So, what we have to do? We have
to compute the; this original coefficient
matrix A which was given for the system of
equations. We need to just compute the lambdas
and the eigenvalues eigenvectors and then
we can find a solution.
So, just to demonstrate this we have taken
the simple example here. We can rewrite in
the system form here. So, dx 1 dt, dx 2 dt
we can write down in this vector form this
coefficient matrix 3 2 7 and minus 2 and this
x 1 x 2. So, what we have to do? We have to
just compute the eigenvalues and eigenvector
of this matrix here.
So, the eigenvalues of this matrix are coming
as when we write down this characteristic
polynomial we are getting this minus 4 and
5 as the eigenvalues and the corresponding
eigenvectors we are getting here 2 7 and 1
1.
So, 1 1 is corresponds to 5 and minus 4 and
corresponds to this 2 and minus 7. So, now,
we can write down the solution directly in
terms of the eigenvalues, eigenvectors. We
have the constant term we need to put this
eigenvector and e exponential power this lambda
t, again the second constant, the second vector
and exponential 5 t.
So, what we have seen here that with the help
of this diagonalization; this was very easy
to solve such a system, and the conclusion
here is that this diagonalization of the matrix
we have learnt, and in particular we have
seen these two applications, the power of
the matrices, we can easily compute with the
help of this idea and also the solution of
the system of linear differential equations
we can compute with this diagonalization.
So, these are the references we have used
and thank you for your attention.
