Hello everyone so welcome to the 3rd lecture
of this module So in this lecture we will
learn a method for computing eigenvalues and
here the method will be a numerical methods
because so far you learn how to calculate
eigenvalues from as a root of characteristic
polynomial So here what we will do We will
apply some sort of similarity transformations
on the given matrix such that after a sequence
of similarity transformations the matrix convert
into a diagonal matrix and from the diagonal
matrix we can see the eigenvalues directly
as the diagonal elements
Furthermore the sequence will also contain
the information about the eigenvectors of
the matrix So this method is called Jacobi
method and this method gives a guarantee for
finding the eigenvalues of real symmetric
matrices as well as eigenvectors for the real
symmetric matrix So as I told you it is based
on the sequence of similarity transformations
and those transformations will be based on
the rotation matrices or given rotations we
will apply the rotation matrices in terms
of similarity transformations to the given
matrix in such a way that all the off diagonal
elements become zero after a series of transformations
Here off diagonal elements become zero means
there should not be any change in the eigenvalue
of the matrix and that is why I am telling
you the similarity transformations because
the two similar matrices will be having the
same spectrum
So first of all here we should know or we
should have the idea about a rotation matrix
So what we mean by a rotation matrix So a
2 by 2 rotation matrix in a plane is given
by by this particular matrix So it is an orthogonal
matrix you can check uhh the having determinant
as one A transpose will be equal to A inverse
and so on This is about the rotation in a
plane by an angle theta If I talk about the
rotation in the space then first of all we
should decide where I want to make the rotation
whether the rotation about X axis or Y axis
or Z axis The second thing by which angle
So if I want to a rotation about X axis by
an angle phi1 then the rotation matrix will
be (1 0 0) (0cos phi sin phi) (0minus sin
phi cos phi) if I want to make the rotation
about Y axis by an angle phi2 so it will become
so the rotation matrix will be this one and
if I want to make it about Z axis by an angle
so I want to make it about Z axis by an angle
3 uhh phi3 then the rotation matrix will become
the third column will become (001) 3rd row
will become (001) and here rotation will be
cos phi sin phi3 cos phi3 minus sin phi3 and
cos phi3
So these are some examples of rotation matrix
in 2D and 3D space 2D plane and 3D space but
suppose a n by n matrix is given to us then
a n by n rotation matrix let us define denote
it by J(p q theta) So here we want to put
the cos and sin terms in pth rows and qth
row pth column and qth column and this will
be of the form this one So except these two
rows rest of row will be like (1000 0100 like
that in this two rows we will be having the
terms of cos theta and sin theta like here
in pth row I mean 0000 then at the diagonal
of pth row and pth column it will be cos theta
at the intersection of pth row with qth column
it will be sin theta Similarly at the intersection
of qth row pth column will be minus sin theta
and the intersection of qth row with qth column
will be having term cos theta
So let us denote this cos theta by C and sin
theta by S then the matrix J(p q theta) is
known as Jacobi rotation matrix or given rotation
matrix The matrix j(p q theta) is applied
to symmetric matrix A as a similarity transformation
and once we applied to A this will rotate
row and columns p and q of A through an angle
theta so that (p q) and (q p) entries become
zero So these are two of diagonal entries
(p q) and (q p) and these two entries will
become 0 and I told you this method uhh method
is applicable only for symmetric matrices
So hence (p q) will be (q p)
So let us denote this the similarity transformation
like this we are having pre-multiplication
of J transpose and a post-multiplication of
J with matrix A and I am getting my next matrix
A dash So A dash is J transpose A into J where
J is a rotation matrix with the p and q and
by an angle theta So let off (A) and off (A
dash) be the square root of sum of squares
of all off-diagonal elements of A and A dash
respectively Then off (A) square will be frobenius
norm of (A) muns uhh square of the frobenius
norm minus i equals to 1 to n aii square So
what we are doing we are taking square of
all the elements and we are subtracting diagonal
elements Since the frobenius norm is invariant
under orthogonal transformations and only
p and q columns are reformed matrix A dash
So we can have the sum of squares of off diagonal
elements of matrix A dash equals to square
of the frobenius norm of A dash minus square
of the diagonal elements of A dash
This equals to a square of the frobenius norm
of A minus i not equals to pq a dash ii square
because there will be change only in p and
q elements pth row (p q) and (q p) elements
minus a dash pp square plus a dash qq square
this will become this particular term and
finally it comes out that the square sum of
the squares of off diagonal elements of A
dash will be less than sum of a square of
off diagonal elements of A it means the square
elements are uhh the elements from off diagonal
are eliminating going towards zero and this
is a basic motivation for the Jacobi method
So here is shows that the size of off diagonal
per decrease is by applying Jacobi transformation
The post-multiplication of A by J1 will change
in columns p and q in the same way the pre-multiplication
of J1 transpose bring changes in rows p and
q Hence the transformation A dash equals to
J1 transpose A j1 alters only rows p and q
and columns p and q of A and there is no change
in rest of the rows and columns
If we see this transformation in n by n set
up so what will be the relation between matrix
A and A dash So the elements said a jk of
the matrix A dash are given by the formula
So when j not equals to p and j not equals
to q a dash jp will be C times ajp please
not that here C is cos theta minus Sajq Similarly
a dash jq is given by Sajp plus Cajq when
j not equals p and j not equals to q The diagonal
entry in the pth row is given by a dash pp
equals to C square app plus S square aqq minus
2 CSapq Similarly the diagonal entry in the
qth row is given by S square app plus C square
aqq plus twice of c into S into apq The elements
on the intersection p th row and qth column
or qth row and pth column is given by that
is a dash pq c square minus S square apq plus
CSapp minus aqq and please here note that
the last and rest of the elements we can find
by the symmetry but here please not that this
element This is the off diagonal element in
the pth row and qth column or qth row and
pth column and we want to make them zero So
if I make them zero I can write zero equals
to C square minus S square apq plus CS app
minus aqq like this Given by equation 2 moreover
the goal of every step of Jacobi equation
is to make the off diagonal elements a dash
pq and a dash qp zero So from this we can
have phi equals to cot 2 theta C square minus
S square So it will be cos 2theta minus twice
of cos theta into sin theta that is sin 2theta
So this is phi
Now from equation 1 and 2 I can write C square
minus S square upon CS equals to aqq minus
app upon apq So from here I can write phi
equals to instead of this I can write this
term So aqq minus app upon twice of apq So
this is equals to cot of 2 theta so tan of
2theta will become this value uhh 2apq upon
aqq minus app and theta will become 1 by 2
into 1sintan inverse twice apq upon in the
denominator we will be having aqq minus application
however a less round of error generated if
we use tan theta in computations like let
us assume t equals to tan theta which is sin
theta upon cos theta So from equation 1 that
is my equation 1 So I divide the numerator
and denominator by C square so it will become
1 minus S square upon C square upon twice
of S upon twice of S upon C So 1 minus t square
upon 2t because S upon C is t and tis gives
a quadratic equation t square plus 2t phi
minus 1 equals to zero
The roots of this quadratic equation is given
by minus phi plus minus square root of phi
square plus 1 and that will be sign of phi
upon absolute value of phi plus square root
of phi square plus Here sign of phi is 1 when
phi is non-negative and it is minus 1 when
phi is a negative number Thus once we get
t we can calculate C and S by this formula
C will be 1sign upon square root of t square
plus 1 and S will be C times t Let us take
an example of this method Consider the matrix
this So this is a 3 by 3 matrix having elements
(1 square root 2 2)( root 2 3 root 2) (2 root
2to 1) and let us solve this matrix or let
us apply
So let us take a 3 by 3 matrix and find out
the eigenvalue of this matrix as well as eigenvector
using Jacobi method So matrix is (1 square
root 2 2) (square root 2 3 square root 2)(2
root 2 and finally 1) So it is a real symmetric
matrix Now first of all in Jacobi method we
will look for the off diagonal element having
the maximum absolute value because we will
perform a similarity transformation to make
it 0 So if I look at the off diagonal elements
the biggest off diagonal elements is this
one that is a31 equals to a13 equals to 2
It means my p is 1 q is 3 So it means app
is 1 aqq is 1 and apq equals to aqp equals
to 2
So here if I calculate theta theta will be
1 by 2 tan inverse twice of apq upon aqq minus
app and it is coming out pi by 4 because here
it will be 0 in the denominator So tan inverse
infinity and so it will be pi by 2 into 1
by 2 it will become pi by 4 This is one of
the way of calculating theta we can calculate
it using the t that quadratic equation first
by calculating phi then t and then from t
we will directly calculate cos theta and sin
theta as I told you Basically that is more
uhh accurate compare to this one Now so define
matrix J So J will be cos pi by 4 that is
1 by root 2 0 that is cos pi by 4 minus sin
pi by 4 sin pi by 4 and cos pi by 4 This is
J1 Calculate A1that is J1 transpose A into
J1 and it comes out 3 2 0230 and finally 00
-1 So please compare A and A1 just see these
two elements I have made these two elements
0 okay just by applying this Jacobi rotation
Now to make this particular matrix as a diagonal
matrix what I need to do I need to perform
make these two elements are also zero So for
making these two elements 0 again I will do
that so my p is 1 and q is 2 and a12 equals
to a21 equals to 2 that is apq equals to aqp
equals to 2 app equals to aqq equals to 3
these two diagonal elements again if I calculate
theta theta again will be 1 by 2 into tan
inverse twice of apq that is 4 upon 0 So it
is coming again pi by 4 and my next matrix
will become J2 So J2 will become (1 by root
2 -1 by root 2 0) (1 by root 2 1 by root 2
0)(001)
If apply again this particular similarity
transformation on A the matrix obtain in this
one I got the matrix A3 A3 will 
be (500) ( 010) and (00-1) So please look
at A3 A3 is a diagonal matrix and hence the
eigenvalue of A3 is 5 1 -1 and so these are
the eigenvalues of A because I obtained A3
just by applying the two similarity transformations
on A Now the eigenvector of this matrix A
is given by the product of J1 into J2 and
I will tell you while I am writing So eigenvectors
will be columns of this matrix that is J1
into J2 So I got the eigenvalue I got the
eigenvectors and hence I will be able to solve
this particular problem for finding eigenvalues
and eigenvectors using Jacobi method
Now actually what is happening let me tell
you few things about this method Let me write
a 2 here which I obtained just by applying
J1 transpose A J1 on A So it was something
like (3 2 0) (2 30) (00 -1) So from here I
can here and from here I can here Now if you
remember previous lecture there we talk about
gershgorin theorem So let us see how eigenvalues
are changing or gershgorin circles are changing
in different iterations of Jacobi method So
if I talk about this and this uhh first disc
is having center at 1 and radius is 2 plus
root 2 so something 3.4 so 2 3 4-1-2-3-4 So
this will be the disc from the first row from
the second row center at 3 and radius is 2
times root So it will be something radius
at 3 so 1 2 3 and center at 2 times root 2
so something 2.8 So it will be like this and
the third row is again center at 1 radius
is 2 plus root 2 that will be just your first
disc and as I told you eigenvalues are 5 1
-1 So one of the eigenvalue will be here at
5 one will be here 1 another one at -1
So gershgorin theorem holds for this matrix
however here what I am having one of the eigenvalue
is the common area of two disc The gershgorin
disc are not disjoint here If I say in this
matrix So let us say 1234 -1-2-3-4 So first
is lambda minus 3 so center at 3 and radius
is 2 So this one so 1 345 second is giving
center at 3 radius 2 which is similar to first
one and the third one is having at center
at minus 1 and radius is 0 So in the second
matrix which I got in the first iteration
of Jacobi method what is having the disc as
overlapping each other and one is disjoint
Now see the third matrix that is A3 Here simple
I am having one of the eigenvalue is here
at 5 another one at 1 and the third one at
-1 and these are the gershgorin disc for the
given problem So basically what I am doing
by applying the Jacobi rotations to the given
matrix I am reducing the size of gershgorin
disc in such a way they become disjoint or
they to a single point that is your diagonal
matrix we are getting here Another thing I
want to tell you here why I am saying that
the product of J1 J2 like that will give me
the eigenvectors What is happening let us
say A1 is the eigen matrix which is having
diagonal entries as the eigenvalue and this
I am getting just by applying a Jacobi rotation
on J
Now as you know that J is an orthogonal matrix
it is a rotation matrix So J transpose will
be equals to j inverse So what happens if
I multiply both sides so it will become 
I can write like this or this is identity
So A into let us say x1 x2…xn are the eigenvectors
of A and as I told you they will be the columns
of matrix J So I have written like x1 is the
first column of J x2 is the second column
of J xn is the third column of J Now look
here what are these A1 is a diagonal matrix
having the eigenvalues of A So it will be
something lambda1 lambda2 lambda n this matrix
into again J so J is x1 x2 xn So when I multiply
this with first column what I will get Ax1
equals to lambda1 x1 which is the first eigenvalue
and corresponding eigenvector for A from the
Ax2 will be lambda2 x2 Axn will become lambda
n xn So hence this matrix J is having eigenvectors
of A as its column which I claimed earlier
that you can see from here
So in this lecture we have learn a method
for calculating eigenvalues and eigenvectors
just by using the similarity transformations
or a sequence of a series of transformations
and those transformations are formed just
using the rotation matrices in such a way
that the off diagonal elements become zero
In the next lecture we will learn another
method for finding the largest eigenvalue
and its corresponding eigenvector for a given
matrix and that particular method is called
power method So thank you very much for this
lecture by see you in the next lecture
