Today 
we will review, what we learnt about Eigen
values, the calculation of Eigen values in
the last lecture and then go ahead and look
at, so then otherwise, are computing these
Eigen values and Eigen vectors.
First, we saw that Eigen values problem is
basically, finding the solution of an equation
A minus lambda I equal to 0 for matrix of
form A. A system of matrix equations Ax equal
to b. In a matrix equations Ax equal to b,
the right hand side of these equations is
0 and then how do we solve that is why we
have started this Eigen values problems and
the importance of solving equations where
the right hand side of this matrix equations
is 0, that is where we started the discussion
on the Eigen value problem and we went around
and said that would be interested in solving
equations of this form that is A minus lambda
I into x equal to 0 that is Eigen values equations,
and we said that this has this kind of equations
have a non-trivial solution, provided the
determinant of the a minus lambda I is 0 that
leads us to the what is called the characteristic
polynomial that is determinant of A minus
lambda I equal to 0 of the matrix of the form
A is the characteristic equations for the
Eigen values.
So this characteristic polynomial, we learned
a way of constructing this, that is using
the Faddeev Laverrier method. We saw that
how to construct given a matrix A minus lambda
I, itÕs characteristic polynomial which is
lambda n minus p n minus 1 lambda n minus
1 etcetera. So we saw that this can be constructed
by finding the traces of the series of the
matrices given by recursion relations starting
that fact that p n minus 1 is just the trace
of the matrix A and then, we can construct
matrices of A into the Bn which is Bn was
the first start with A itself, so it is A
into Bn minus pn minus 1 I. We can construct
matrices of that form and take the traces
of that and as the traces which goes as the
coefficients of this polynomial of this characteristic
polynomial.
So we saw that and then, we were wondering
how to solve the equations, now we have the
polynomial but we have to find the roots of
this polynomial. This is the polynomial in
lambda. So the roots of this polynomial are
our Eigen values. So the Eigen value, finding
the Eigen value, the problem of the finding
the Eigen value was then that way splitted
into two, one is to find the characteristic
polynomial which we said for a large matrix
is finding the characteristic polynomial itself
is a non trivial problems, and we say that
we would use this method, use this which we
just mentioned the Faddeev Laverrier method
for that. And then once, we have the characteristic
polynomial, we saw a program also in the last
lecture, of constructing the polynomial coefficients
and we have the characteristic polynomial
then the question is how do we find the roots
of that characteristic polynomial and so now,
that leads us to a general problem of finding
the roots of a n th order polynomial and we
said that one way to do that is to use what
is called the BairstowÕs method.
So in the context of the Eigen value problems,
it is the solution of the characteristic polynomial,
or even in the more general sense, it is the
solution of n th order polynomial which are
the roots of the n th order polynomial which
is want to be find out, all the zeros of n
th order polynomial or an equations, roots
of an equation of this form that something
on the left hand side equal to a0 plus a1
x plus a2 x square up to an xn. So the BairstowÕs
method applies a technique of this form that
is we say that, we would find roots of the
this polynomial in pairs of conjugate variables
that is, we allow for the roots to be complex,
this n th order polynomial could have complex
roots then, we get them in pairs such that
we have the roots as complex conjugate and
that is general enough, if the coefficients
of this polynomial is real then that is general
enough that is we do not lose anything by
making this particular, taking this particular
approach.
So, if you want to find roots of this polynomial,
n th order polynomial in pairs, that is we
want to allow for complex roots and then,
the complex roots come in conjugate pairs
and then, best method is to and that is what
BairstowÕs said is to divide this polynomial
by quadratic polynomial or to factorize this
polynomial as the product of a quadratic polynomial
multiplied by n minus 2 order polynomial right.
So that is what we discussed in the last class.
So we said that fn the n th order polynomial
can now be written as x square minus r x minus
s into a polynomial of order n minus 2 which
is now given by bn x n minus 2 plus b n minus
1 x n minus 3 up to b 2. So then, if you take
an arbitary value of r and s then we will
have a remainder to this, that is b1 x minus
r plus b0. Now the advantage of this method
is that we can determine given a r and s,
we can determine this coefficients b1 b n
minus 1, b2, b1, b0 etcetera by relation of
this form. So that gives us a great advantage
that, so we have the polynomial whose n th
order polynomial, whose coefficients were
given by anÕs and then we divided that by
x square minus r x plus s and we got the coefficients
of polynomial of order n minus 2 and order
of n minus 2 whose coefficients are bn to
b2, so that is given here. And then, we have
a remainder which has the b0 and b1 which
is given by b0 b1 into x minus r plus b0.
So that is what, so the idea is given the
anÕs we can even determine the bÕs. So the
question is how do we determine r and s, so
then we said that, we could do a kind of an
iterative approach that is we could say that,
we start with r and s and we will compute
what the remainder is which will be given
by b1 x minus r plus b0 and then, we would
adjust just the r and s such that the remainder
is 0, so that is the idea.
So for that what we do is, we assume this
and we start with a guess value of r and s
and we look at using the relations which I
have just now solved that is using this relation
we could compute b0 and b1 that is b1 would
be a1 plus r b2 plus s b3 and b0 is a0 plus
r b1 plus s b2 and we could see the value
of b0 and b1 whether they are 0 or not if
they are not 0 then we could say that okay
I will expand this b0 and b1 and around the
value r and s which I just assumed.
So when I write b1 at r plus delta r, s plus
delta s as b1 plus del b1 by del r into delta
r plus del b1 by del s into delta s and similarly,
b0 at r plus delta r, s plus delta s as b0
plus del b0 by del r into delta r plus del
b0 by del s into delta s. I can write it like
this and then I would say that okay, so the
new value of b1 r plus delta r, s plus delta
s at r plus delta r, s plus delta s and b0
at r plus delta r, s plus delta s should be
0.
So I will find the value of delta r and delta
s such that, this is 0 so this left hand side
of the equation is equal to 0, I could find
the delta r and delta s, such that this is
satisfied so that is what the approach was,
so that would lead us to an equations of this
form that del b1 by del r into delta r plus
del b1 by del s into delta s is equal to minus
b1 and del b0 by del r into delta r plus del
b0 by del s into delta s equal to minus b0
and this is a simple linear equation and we
can solve is by elimination immediately. So
now provided we can do that, provided we know
what this derivates are, del b1 by del r,
del b1 by del s, del b0 by del r, del b0 by
del s etcetera, so provided we know that this
derivatives are, we can determine delta r
and delta s.
So we assume r and s and we compute b0 and
b1 and if they are not zero what we do is
we expand the value b1 and b0 around r and
s and find out the delta r and delta s such
that the new values of b0 and b1 are 0. So
that is what the approach is. So when we actually
expand that of course we take keep terms only
up to the first order in delta r and hence
the new value of delta r and delta s we get,
may not make the b1 and b0 as 0 right. So
when we expanded this here, we have expanded
it only up to first order in delta r and delta
s. So this value may not go to 0, even if
you solve this equation this right hand side
equation with putting b1 at r plus delta r
and s plus delta s as 0 that is, if you solve
this equation the resulting delta r and delta
s may not make b1 and b0 as 0, because we
terminate it at first order but then we could
do this iteratively and then obtain values
of delta r and delta s such that b1 and b0
as 0.
So now the question was how to determine these
derivatives. So for that BairstowÕs showed
that the way we constructed this b1 b n minus
or all the bnÕs or all the biÕs from the
aÕs, we could use the similar method to construct
the derivatives of the form del b1 by del
r, del b1 by del s etcetera from the b1Õs
that is, we said that ok we could write a
recursion relation of this form that is cn
is equal to bn, c n minus 1 equal to b n minus
1 plus r cn etcetera, a form and then we could
take the derivates of del b1 by del r and
del b1 by del s from this equations, that
was the idea ok.
So we would get this equations as that is
del b1 by del r and del b1 by del s 
as you know has c2 and so del b1 for example
del b1 by del r, so b1 here would be b1 plus
r c2 right. So del b1 by del r will get us
c2, so we could substitute that here ok c2,
c3, c1, c2 etcetera that is del b1 by del
r, del b1 by del s, del b0 by del r, del b0
by del s from this relations so we have the
aÕs is and then we have the bÕs is that
is aÕs are the coefficients of the n th order
polynomial and bÕs were the coefficients
of the n minus 2 order polynomial and then
we construct this cÕs from the b and that
is just to get us the coefficients that is
the derivative of del b1 by del r del b1 by
del s etcetera.
So once we have that, we can solve this equations
and obtain the delta r and delta s to our
satisfaction that is, satisfaction in the
sense that we put a tolerance for the value
and then get the b1 and so get the values
and we get the delta r and delta s such that
b1 and b0 are 0. So that is the method, so
we just see an implementation of this method
in this code here.
So we had a set of coefficients, the set the
polynomial which has the coefficients of that
of this form, okay that is the coefficients
of this polynomial we have, so let me I just.
So we will just see this as the coefficient
of the xn th term, this is the coefficients
of the x to the power of the 0 term and this
the coefficients of the last one is the coefficients
of the n th power term. So this is actually
the coefficients of x to the power of 5, okay
that is a5 and then this coefficients a4,
a3, a2, a1, a0. So this we are solving 5 th
order polynomial here. So we are solving to
start with a fifth order polynomial here.
Okay, whose coefficients are a5 as 1, a4 minus
3.5, a3 2.75, a2 2.125 and a1 minus 3.875
and a0 1.25 or in short it will be x to the
power of 5 minus 3.5 times x to the power
of 4 plus 2.75 times x to the power 3 plus
2.125 into x to the power of 2 minus 3.875
x plus 1.25 equal to 0, that is what you trying
to solve. So that is the polynomial which
we have.
Now we are trying to factorize this polynomial
and get the roots. So that is what we have,
so that is, fist this program ok started here
and I defined all the appropriate variables
here, all floating point and then I open this
file which I just showed you coefficients
dot data coeff.data and then read out all
the coefficients using this fscanf function
I read out all the coefficients into this
array a. So once I have all this coefficients,
so then I know the order of my polynomial.
So this n gives me the order of my polynomial
and n equal to 4 means 0 to 4 that is 5.
Okay so then what I have to do, I have to
now iterate this right, I have to now do this
calculation here, ok so I have to construct,
I have the an matrices so I have the an arrays
given by just now you show that I have the
polynomial an, okay so that is all the a0
to a5 I have that is a0, a1, a2, a3, and then
a5 and then from that now I need to construct
the bnÕs so that is the first step so that
is done here. Okay, so as long as the order
of the matrix, order of the polynomial is
greater than 2, I have to construct I have
to continue this iteration till the order
of the polynomial is 2 right. So first I take
the 5 th order polynomial and I divide the
5 th order polynomial by x square minus r
x minus s that is this example we are looking
at, we took a 5 th order of polynomial and
now I am going to divide that by x square
minus rx minus s and then I have to construct
now b5 to b2, b5 ,b4, b3 and b2. I have to
construct that and that is what is done here.
So I put m equal to n here and say m is the
same as n here right now that is greater than
2. I constructed, I use this, I use this relation
that is bn and an, b n minus 1 equal to a
n minus 1 plus r bn and bi, for i going from
n minus 2 to 0 is given by ai plus r b i plus
1 plus s b i plus 2 so that is it.
Ok that is what this whole thing is doing
here right, that is first the bm equal am
and b m minus n equal to b m minus 1 plus
r bn and then I go and construct all the polynomials.
So I did that, so j going from m minus 2 to
greater than 0 and I come out here with this
polynomial with all the b value. So I have
all the b values here, ok now once I have
the b values and then I now construct the
c values right, so now the next step is to
find out the del b1 by del r and del b1 by
del s etcetera. So that I do that by, so I
know the del b1 by del r, del b1 by del s,
to construct that I need the c relation and
then I can say del b1 by del r is c2 and del
b1 by del s is c3 etcetera, so from this relation.
So that is what I am going to do. So that
is for that cÕs has to be constructed and
here c is constructed. So once I have cÕs
here okay, this is exactly the same as bÕs
here but now all the bÕs here are replaced
by a and all the aÕs are replaced by b so
that is the c constructions and then I have
the matrix this 2 by 2 matrix which is c2,
c3, c1, c2. So that matrix and right hand
side is minus b1, minus b0 remember that and
that is what is constructed here, so the 0,0
element is c2 the 0,1 element is c3 and 1,0
element is c1 and the 1,1 element c2 and then
the right hand side is given by the d column
vector here that is minus b1 and minus b0.
So I have all the matrices and I just saw,
I invert this, I solve this particular equations
that is c2 c3 matrix c2, c3, c1, c2 into delta
r, delta s equal to minus b1, minus b0 that
by inverting this matrix c2, c3, c1, c2 and
multiplying it to b1 and b0 that is what done
here. You have seen this in the case of non-linear
function fitting a non-linear function to
a set of linear points is exactly the same.
So we have this relation here and now, we
use that particular r and s, delta r and delta
s value which we obtain by actually multiplying
the inverse of this D matrix into this d matrix
this little d matrix that is our d1 and d0
and I get a delta r and delta s and I say
r equal to r plus delta r and s is equal to
s plus delta s.
So I just short cut one step here, I just
straight away write delta r as an expression
here and then, what is the r and s values
I am getting what is b0 and b1 in that step
is and that I print that out here. Okay this
print statement prints b0, b1, r and s, okay
so that is what this print statement prints
out, so it actually printouts the order of
my polynomial, the r value ok that you can
see here in the print statement. It will print
out the order of the matrix, order of the
polynomial in that step the r value, the order
to begin with in that step will be 5 right
the r value, the s value, the b0, the b1.
So once we have got the b0 and b1 we go back
here and check what is the b0 and b1 values
are, we want it to be 0.
So I have given here a tolerance of 0.001.
So b0 and b1 are not 0 they will go continue
with new value, it will continue the iteration
further right, so it will continue the iteration
till, so now the new c values will become,
this actually does not it can even be outside
this loop, this does not change because aÕs
are not changed in this loop, okay but the
bÕs are changed. So the new cÕs will come,
so we continue this iterations ok this are
changed because r and s are changing okay.
So the bÕs will change because r and s will
change the new r and s, with the new r and
s value we will calculate b and c and we will
continue the iteration till we obtain a new
delta r and delta s and we will continue that
till b0 and b1 are 0. Okay that is the first
step.
So once we have done that, we have found the
solution, we have the found the r and s okay,
then what have to do, we have to know find
the solutions. So now once we have found that
we have found one factor right, so we have
found the factor of the form x square minus
r x minus s, so we have found the r and s
value which factorizises our n th order polynomial
to n minus 2 th order polynomial multiplied
by x square minus r x minus s. So once we
have that we know that solution of this equation
straight away as r plus or minus root of r
square minus 4 s divided by 2. We do the root
of the equation, so that is what we use. So
we just want to find out r squared, so this
is minus here so it is r squared plus 4 s,
we have to find out the solution of, we have
found r and s that means we have found a factor
of the polynomial that is x square minus r
x minus s equal to 0 and that the solution
of that is x equal to r right, plus or minus
root of r square plus 4s because there is
a minus here. So plus 4 s divided by 2, so
remember we have found r and s values so we
are finding the, what is the quantity r square
plus 4s, so r square plus 4s can be negative
okay, because we allow for complex roots.
So when you say x square minus r x minus s
equal to 0 the roots of that quadratic equations
can even be a complex number. We allow for
that, the complex conjugates, ok we say r
plus or minus root of r square plus 4 s by
2. So r square plus 4 s can be negative that
will give us a complex conjugate pair. So
we have to see if that makes sense in the
c program, we cannot handle complex number
that way, so we have to first find out whether
r square plus 4s is negative or not. If it
is positive we could straight away take the
root of that and write the solutions here.
I have written the solutions here as r plus
square root of x where x is now x square plus
4 s remember okay. So r plus square root of
x by 2 and r minus square root of x by 2 as
my 2 solutions. If this is negative then we
have to take the root with other sign, that
means we should find the square root of minus
x and add a i in front of it okay, so that
is, so now the solution will be r plus i square
root of minus x and r minus 
i square root of minus x.
So that is what is written here. So now the
real part of r by 2 the imaginary part is
square root of minus x by 2, okay I treat
them separately now since C does not allow
for complex numbers. Okay, so r by 2 and square
root of minus x by 2 and r by 2 square root
of plus x by 2 sorry minus x by 2 are two
variables plus and minus. So that is what
I am going to print out. Okay so then once
we have finished that okay, so we will then
come to, we will have what we left with, that
is the quotient polynomial and we factorized
it okay we divided the n th order polynomial
by x square by r x minus s and now what is
left with is n minus 2 th order polynomial
right now, this case we had the 5 th order
polynomial. So we have the third order polynomial
that is quotient polynomial.
So I print out here, the coefficients of that
quotient polynomial, okay and that is it will
be printed out here and that it will be now
remember a j is, so what goes as a0 of the
quotient polynomial right would be the b2
correct, that is the way we have written our
polynomial remember. So we have written our
polynomial as this way, that is the quotient
polynomial is given by this one, that is the
x to the power of 0 order term coefficients
is b2 so that is what I call a0 right the
original polynomial, so in the program I have
to replace all the a values with now the new
a values after for the question polynomial,
so that is a0 becomes b2 and similarly, we
go all the way to a3 will now be b5. So that
is what is here, so n is now was 5 in our
case. So 5 minus 2, so the highest power in
our case now will be x cube and the coefficients
of that would be b5.
So once we have all the bÕs, the correct
b is constructed, we can get the quotient
polynomial right from this and we can go back.
So I constructed the quotient polynomial and
then I simply go back here. Okay and then
I continue the process. I now reduce my order
by m as m minus 2 and then we will continue
that and then I will get a new polynomial
a cubic polynomial and again I factorize that
cubic polynomial as a quadratic into a linear.
So then we find the, we know the solutions
of the quadratic polynomial linear, so we
find the linear solutions is the quadratic
polynomial and then we can read out the solution
from the linear program that is what we do
here. Okay and we can see that here, this
implementation of this, I will show you, so
you have seen the coefficients data and I
now we will see. So okay.
So when I run this program, there is one more
step which I forgot to tell you, that we have
to give some initial guesses of the r and
s okay that is given here. Okay so once we
have read out all that a coefficients we will
have to give an initial guess for r and s.
Okay, so we have to give the initial guess
for r and s and with that initial guess we
will go ahead and solve for the correct quadratic
polynomial that is the correct r and s values
and then we have n minus 2 th order of polynomial
left and we have to factorize that for that
we will use the newly obtained r and s as
the initial guess and continue.
So we have to give only one guess, so we could
have a tenth order polynomial but we have
to give only one guess value of r and s and
then will continue use that for the next lower
order polynomial, so that is what we are doing
here. So we had one guess for r and s has
to be given r and s and such the when we saw
the program it will wait for this input and
then in the next round after it finishes the
finding the r and s value prints out the solutions
and then it actually goes back here right,
so when it is goes back in this m equal to
m minus 2 and then it goes back okay. So when
it does that, when it goes back from here,
okay it now does not ask you for new r and
s value, okay the r and s value is outside
this loop but it takes, whatever the value
r and s values obtain for the previous factorization
as the guess for the next one.
So that is what is here. So we start with
some guess value let say minus 1 and minus
1, I have given minus1 and minus1. So then
it will, so now it is printing out the order
of polynomial which is 5, I have given minus
x, minus 1 as the starting value for r and
s but it has computed new r and new s which
is minus 0.644 and s is 0.138 where the b0Õs
are far from being 0 right, so b0 is 11 and
b1 is minus 10. Okay, so obviously my guess
was pretty bad, okay it goes back and then
compute for the new r and s as the guess and
again delta r and delta s have computed new
r and s so that turns out to be minus 0.5
and 0.4 and the b0 was reduced to 2.1 and
b1 as minus 1.80. As again still far from
0 so it will continue iteration, okay and
then finally comes to b 1 0 and b 0 0. So
it converges pretty fast and then it is gone
to r as minus 0.5 and s as plus 0.5, so that
is the r and s value for this.
So we have one factor that is x square plus
half x minus half as one factor right, as
is r is minus 1/2 and s is plus 1/2 and corresponding
solution are x equal to 1/2 and x equal to
minus 1 right, x square plus 1/2 x minus 1/2
equal to 0 or 2 x square minus x plus 1 equal
to 0 so x equal 1/2 so x equal to minus 1
are our solutions.
So then we have now left with an m equal to
3 polynomial right, that is what we are left
with. The coefficients of that x cube term
is 1 remember x cube term coefficient is the
coefficients of the x the power of 5 term
earlier, so x cube term is 1 as we have the
polynomial which is left as x cube minus 4
x square plus 5.25 x minus 2.5, that is polynomial
which is left.
So now I start with r equal to minus 0.5,
s equal to plus 0.5 as initial guesses and
I compute the new r and s values from the
iteration. Again we can see that the b0 and
b1Õs are pretty bad to begin with and then
it converges, it converges to b0 equal to
0 the b0 equal to minus 0.000021 and b1 equal
to 0.000055 which is much less than the tolerance
I have given. I have given it as 0.001 as
the tolerance. So you got a new set of r and
s values, r equal to 2 and s equal to minus
1.25. So now, the new polynomial, new factors
that is x square minus 2 x plus 1.25 equal
to 0 that is solutions we have to find. Now
it turns out that here r square plus 4 s,
r square is 4, 4 s is 5 minus 5, so r square
plus 4 s is negative.
So our roots are now complex numbers, so one
root is 1 plus 1/2 i and other root is 1 minus
1/2 i, obviously because they are complex
conjugates and we have a quotient polynomial
which is now linear which is x minus 2 for
1 or minus 1.999984 which is 2. So x minus
2 equal to 0 or x equal to 2 is a next solution,
so we found all the roots of the polynomial
this polynomial. We found one root as 2 and
one as 1 plus 1/2 i and other as 1 minus 1/2
i and we found earlier root as 1/2 and minus
1. So, 1/2, minus 1, 1 plus 1/2 i, 1 minus
1/2 i and 2 are the different polynomials,
different values of the different roots which
we obtained from this.
So we have all the Eigen values. So that is
one method of computing the Eigen values.
They are other ways of doing this may be depending
upon the problem, we will have to choose which
is the fastest method. So we will look at
the some more methods okay, so we just to
summarize what we just saw is to compute the
r and s for each of the polynomials and we
look at the quotient polynomial till it is
quadratic or cubic or linear I should say
ok, so if it is linear I should say which
has the solution as x equal to minus s by
r that is what be done okay and whatever the
quadratic polynomial which we get as a factor.
We just solve that and obtain the solutions.
Okay then we said that this would also be
briefly mentioned in the last class that at
we could, it is not the only way to compute
Eigen values and Eigen vectors. We could use
what is called the vector power method. So
in the power method, we use the fact that
the Eigen vectors for what is called a complete
basis that means that given any vector, so
if you have a n by n matrix given any column
vector, n column vector you can write that
as a linear combination of the Eigen vector.
So any arbitary vector V, I have given V as
the arbitary vector here.
So any arbitary vector can be written as a
linear combination of the Eigen vectors, let
say x1, x2 , x3 up to xn are the Eigen vector
where each of them are column vectors say
x1 , x2, x3 up xn are the column vector which
are the Eigen vectors of the n by n matrix
A that is it. So we now have an idea what
it means, Eigen vectors and Eigen values are.
So let us say these are the Eigen vectors
of this the matrix A and then any arbitary
vector then I can write as linear combination
of this coefficients of this vectors x1, x2
etcetera. Remember each of them are column
vectors, this is a fact and I can use this
fact to construct the Eigen values and Eigen
vectors.
So let us say lambda 1, lambda 2 up to lambda
n are my Eigen values and I have ordered in
such a way that lambda 1 is my largest Eigen
values and then lambda 2 is the next larger
etcetera. I assume that no two Eigen values
are the same or the first or the largest Eigen
values are not the, or no two Eigen value
are the same, I assume that. This will work
only in that particular case, this power method.
Here an assumption I have to make is that
no Eigen values are same or in physics term
you say there is no degeneracy, ok and then
we would write any arbitary vector V0, we
have just taken any arbitary vector V0 and
multiply it by the matrix A, so I have just
taken a arbitary vector V0 which I know can
be written as we just saw a x1 plus a x2 plus
a x3 right which we just saw that can be written
in a arbitary vector as this form and then
I multiply both side by this matrix A, so
I get A V0 and call it I call it V1.
Okay so that V1 is now a1 into A x1 plus a2
remember each of this is a column vector and
this is now n by n matrix right, so each of
this is a column vector and this is n and
by n matrix, so I can multiply this and I
get this vector V1 then I know A x1 by the
Eigen value equations that A x1 is lambda
x1, lambda 1 x1 and A x2 is lambda 2 x2 right.
So I can write this equations ok as, this
equations as a1 lambda 1 x1 plus a2 lambda
2 x2 an lambda n xn and then I take this and
again multiply it by A and take the V1 and
multiply it by A and right hand side again
by A. So I get V2, so V2 will be a1 lambda
1 A x1 plus a2 lambda 2 A x2 plus etcetera
and then but A x1 is lambda 1 x1 so V2 will
be a 1 lambda 1 square x1 a2 lambda 2 square
x2 etcetera or if on repeated multiplication
of this matrix on the right side by A, I will
get V to the power n as A to the power of
n V0 plus a1 lambda 1 power to the sorry this
to be m. So if I multiply this by m times
if I multiply it by m times and then I get
it is Am as Vm as A to the power m V0 as a1
lambda 1 to the power m x1 plus a2 lambda
2 to the power m x2 etc.
So that is multiplied by m times and then
if lambda 1 m if lambda 1 is largest Eigen
value in this right, I could take if I take
lambda 1 m out of this matrix, out of this
equation I will have a1 x1 plus a2 lambda
2 by lambda 1 to the power m x2 plus a3 lambda
3 by lambda 1 to the power m etcetera up to
an lambda n by lambda 1 to the power m xn,
but we said lambda 1 is largest Eigen value
and in that case lambda 2 by lambda 1 is a
number which is smaller than 1 and lambda
2 by lambda 1 to power m is almost 0 right
any number which is less than 1 raised to
the power m which is a large integer is 0
right.
So for example, if the ratio between lambda
1 and lambda 2, lambda 2 by lambda 1 is 0.1
by the time we have multiplied this matrix
by 4, 4 times which is 0.1 to the power 4
which is already very small number. So what
it depends, in short what we find is that
you take an arbitary matrix V and keep multiplying
it by this matrix A and then what we end up
with is that since this term will dominate
everything else, everything else will be negligible
we get that the result of multiplying an arbitary
vector V0, a column vector V0 by the matrix
A, m times, m being the large enough integer
is that you get on the right hand side a1
times lambda 1 to the power m x1 and everything
else is 0. So from this we can read off what
the Eigen value is and what the Eigen vector
is.
So if you have taken an arbitary vector and
multiplied it by matrix A m times and if you
normalize what we got on the right hand side
then what you get is the Eigen vector corresponding
to largest Eigen value and the factor which
multiplies it is the largest Eigen value and
we will see the will see implementation of
this. Okay so this is the in short something,
say that if I multiply it by m times I just
simply get on right hand side of the Eigen
vector corresponding to the largest Eigen
value. Okay so here is a small example of
that.
Okay, so we will see this, that is I have
a matrix A, I want to find the Eigen values
and Eigen vectors. So we will first determine
highest Eigen value or the Eigen vector corresponding
to the largest Eigen value and then we will
see how to generalize this to get all the
Eigen values. So first we just determine the
largest Eigen value.
So here is the matrix, look at this matrix
and then we want to find out now, so we start
with some arbitary matrix, some arbitrary
vector okay, so it is a 3 by 3 matrix, it
is a arbitary vector, I took 1, 1, 1 for simplicity.
So I took 1, 1, 1 for simplicity and I said
I multiplied that vector that matrix by 1,
1, 1, okay and I got something on the right
hand side as 1. 718, 0, 1. 778.
So now I make the largest element on this
column as 1 and so now I have, what I have
got was, I took the matrix A and multiplied
it by 1, 1, 1 and I got some number multiply
by 1, 0, 1. Okay that is my first iteration,
now what I do, I multiply again this one,
I multiply my vector, my matrix, I take this
vector 1, 0, 1 and I multiply it by A again.
That is what I want to do. So that is the
next step, I multiply the matrix by 1, 1,
1. I got 1, 0, 1 ok and then I multiply 1,
0, 1 by the matrix again, the next step the
iteration number 2, I multiply it by 1, 0,
1 and I get something new that is 3.556, minus
3.556, 3.556 that is I make the largest elements
on this column vector as 1. So I take the
3.556 out and then I get 1, minus 1, 1 right.
Now I multiply the matrix by 1, minus 1, 1
right. So at each step I can compute kind
of percentage error right. So the previous
number I got here, the Eigen value was 1.778
and now I got 3.558, so the error is about
the 50 percent, I can continue the iteration
till this error the percentage error that
is the deviation for the previous value for
the next iteration value is very small.
So I repeat this procedure, ok so now I take
1, minus 1, 1 and multiply it by this matrix
A again and I got now minus 7.112 and minus
0.75, 1, minus 0.75. So I guess you follow
what I am doing. I just take an arbitary vector,
I started with arbitary vector multiply it
by the matrix A and I got a vector here, column
vector here. I made the largest element in
the column vector as 1 and then I took that
column vector and then multiply it by the
matrix A again, okay so now we are in the
third iteration of the process and we had
the previous iteration, we had got minus 1,
minus 1, 1 as the vector, we multiply that
by this matrix again and I got minus 0.75,
1 and minus 0.75, minus 7.112. So, okay so
error again so the previous error that is
the value obtained at this step minus the
value obtained at the previous step divided
by the value of this step is the percentage
error, that turns out to be pretty high in
the third iteration, the second iteration
it was only 50 percent, here it is 150 percent
because we have a change of sign. So that
as long as this is large, this is not we are
not obtained our goals, but this may not decrease
monotonically that is what we just saw.
Then the fourth iteration we took that minus
0.75, 1, minus 0.75 we got from the previous
step and then multiply it by the matrix A
and we got minus 6.223, minus 0.714, minus
0.714, now we can see that we are reaching
somewhere ok, we are reaching to values close
enough. We started with minus 0. 75, 1, minus
0.75 and we ended up with minus 0.714, 1,
minus 0.714. If this truly an Eigen vector
you know what we get right, if this is truly
an Eigen vector this product should give you
exactly the same here with number here right,
so that number here would be the Eigen value
Ax equal to lambda x that is our idea in this
iteration scheme and the reason this work
is because we are because the largest Eigen
value dominates, that is what we just saw
right okay.
Okay so the 5 th iteration the 714 had gone
708, so we are almost converging. So we converging
somewhere close to 0.7 ok that is the, so
it is converging and to that Eigen value is
converging on to something like minus 0.607.
So this method this way we could construct
the Eigen vector and Eigen value corresponding
to the largest Eigen value. So this is the
power method, this is the demonstration of
the power method. You take an arbitrary vector
and keep multiplying that by the matrix A.
So now we each time you normalize your vector
to make the largest element in the vector
as 1 and then take that vector and multiply
it by the matrix again ok you drop this and
then when we reach the final answer, when
we actually have the correct Eigen vector
and the matrix multiplied by that Eigen vector
should give us the Eigen value times the Eigen
vector, the vector itself that is the idea.
So now, we could ask the question that can
we generalize this ? Yes we can generalize
this, ok we can generalize this to obtain
even the, even other Eigen values. Okay so
that is the for example, we can very easily,
see how to get the smallest Eigen value. So
now I said that lambda 1, lambda 2, lambda
n was the Eigen values of the matrix A, ok
and now if lambda 1, lambda 2, lambda n are
the Eigen values of the matrix A then we know
that lambda 1 inverse, lambda 2 inverse, lambda
n inverse are the Eigen values of matrix A
inverse. So let us say, if lambda 1 or if
lambda 1 happens to be smallest Eigen value
of the matrix A then lambda 1 inverse would
be the highest Eigen values of the matrix
A inverse and so we could just simply find
the largest Eigen value of A inverse that
will give us smallest Eigen value of the,
inverse of that would give us the smallest
Eigen value of the matrix A and we know the
Eigen vector do not change right, the Eigen
vectors of the matrix A are also the Eigen
vector of matrix A inverse.
So immediately we can see that by using the
power method, we can determine provided we
can compute A inverse which also we have learned
how to do. So provided we can compute the
matrix A inverse, I can compute the Eigen
values, the largest Eigen value and the Eigen
vector corresponding to the largest Eigen
value and the smallest Eigen value and the
Eigen vector the corresponding to the smallest
Eigen value and again computing the matrix
A inverse we could use the Faddeev Laverrier
method which we learnt in the last class.
So, now once this method, so we have this
idea and we can generalize this to get all
Eigen values because now what we could do
is we could construct this method and this
matrix A minus pI where p is, and then new
matrix B minus pI inverse and construct Eigen
value, the largest Eigen value of this matrix
B. Now let see, what this means, so let me
write down this equation and A minus pI, so
A is our original matrix and we subtract some
number times identity matrix from that and
we write down that matrix equation here so
x i is a column vector so we have A x i minus
p x i you know right. But A x i is lambda
x i and p x i is p x i of course and then
px is now just a diagonal matrix.
So this will be just simply lambda minus p
x I, so the p I is a diagonal matrix, so this
would simply give us p x i, p is a number,
so lambda i minus p x i, so we know if I construct
Eigen values of B now that will be lambda
i minus p inverse will give the Eigen value
of that right. So if I can have an idea about
what the Eigen values are then I can take
the number p which is the close to Eigen values
and construct this new matrix B which is A
minus pI inverse and then the largest Eigen
value of this matrix would be something close
something were largest Eigen value of this
B matrix would be lambda i minus p inverse
where lambda i is now the Eigen value of the
A matrix. So the only disadvantage is that
we need to have an idea about where the Eigen
values are, but again there are some ways
of guessing what the Eigen values are. Okay
so we could use for example just use a theorem,
that if you just take the diagonal elements.
Okay, so we could for example just use theorem
that if you just take the diagonal elements,
okay we could just state this theorem that
the Eigen values lies in circle whose centers
are the diagonal elements. So if for example
here it will be Eigen value of this matrix
would be around 4, around 1, around minus
6. Okay and the magnitude it will be around
this value, now how far how far from the 4
a table will be, okay how far from 4 the Eigen
values will be and that the distance will
be given by the sum of the magnitudes of the
other elements in the row i, that is this
will 4 plus or minus 2 and this will be 1
plus or minus 2 and this will be, the third
Eigen value will be minus 6 plus or minus
2.
So I can use that, so I get a p, I can choose
p as 4, 1 and minus 6 and construct the new
matrix as A minus 4 I inverse and look at
the largest Eigen value of that. So that will
give me the Eigen values and Eigen vector
which is close to 4. So in this way I can
construct all Eigen values and all the Eigen
vectors. So that is advantage of power method,
we will get both the Eigen values and the
Eigen vectors by simply, by constructing,
finding the product of a matrix with an arbitrary
vector. So that is the summary of our discussion
on Eigen values and Eigen vectors and next
we will go into again solutions of non-linear
equations then we will continue from the discussion
which we had on the polynomial, the roots
the polynomial. We will go and look at the
solution of others non-linear equations and
that is what we would be discussing in the
next class.
