Hello, welcome to another module in this massive
open online course.
Let us continue our discussion ah regarding
the mathematical preliminaries, for the framework
of convex optimization, ah looking by looking
at another very important concept that is
of ah the Eigen values, ah the eigenvectors
and Eigen values of ah square matrices right.
So, we are going to start talking about the
Eigen the concept of or rather the concepts
of eigenvectors and, Eigen values ok. And,
now these is the eigenvector the notion of
Eigen vector and Eigen value is defined for
a square matrix correct.
So, for a square matrix A all right x bar
is an eigenvector, x bar is an eigenvector,
if we have A x bar that is the product of
the matrix A with the vector x bar, A x bar
equals a multiple of that is lambda times
x bar all right. And this is a fundamental
equation for the eigenvector all right and
well the this lambda is known as the Eigen
value.
This is known as the Eigen value and this
vector x bar is known as the eigenvector ok.
And, now since A x bar equals lambda x bar
that implies A x bar equals lambda, times
I can use the identity matrix here, times
x bar which implies A x bar minus lambda I
times x bar equal to 0, which basically implies
that a minus lambda I times x bar equal to
0. Now, this implies what this means is this
matrix A minus lambda I this is a singular
matrix.
There exists a vector such that a minus lambda
I times x bar equal to 0, which implies that
the determinant of a minus lambda I equals
0 ok. So, we use this to denote the determinate
ok. So, if lambda is an Eigen value of A that
means, the determinant of A minus lambda equals
0. Now, by evaluating the determinant you
can derive an equation that is known as the
characteristic equation corresponding to the
matrix A, and the roots of that all right,
the roots of this equation in lambda give
the Eigen values of the matrix A all right.
So, this gives so this basically gives the
characteristic polynomial correct. So, a minus
determinant of a minus lambda I, this gives
you the characteristic polynomial, polynomial
of a in terms of lambda. The roots 
and the roots of the characteristic polynomial,
these are the the roots of the characteristic
polynomial are Eigen values of A for example.
Let us say A is the matrix 1 1 1 minus 1 let
us take this simple 2 cross 2 matrix ok all
right. So, this is a square matrix A 
and now we want to find the Eigen values ok,
for this given matrix A 2 cross 2 matrix,
we want to find the Eigen values and also
the corresponding eigenvectors ok. So, find
Eigen values 
and the eigenvectors of A.
Now, we given a now let us start by considering
a minus lambda I, that will be equal to 1
1 1 minus 1 minus lambda times 1 0 0 1, which
is equal to 1 minus lambda 1 1 minus 1 minus
lambda all right. And, now we have to consider
the determinant of this, now determinant of
a minus lambda I equals 0, now if you compute
the determinant of a minus lambda you will
see this is this implies 1 minus lambda into
minus 1 minus lambda minus of 1 this is equal
to 0, which basically implies that minus of
1 minus lambda into 1 plus lambda equal to
1.
Which implies that ah lambda square minus
1 equal to 1, which implies that lambda equals
plus, or minus square root of 2. So, these
are the Eigen values of A lambda equals plus,
or minus square root of 2, these are the Eigen
values of the matrix A ok. So, we have got
2 Eigen values that is lambda equals either
plus square root of 2, or minus square root
of 2 ok. Now, let us find the corresponding
eigenvectors of the matrix A corresponding
to both these Eigen values ok.
Now, the eigenvector to find the eigenvector
ok, what we are going to do is we have 1 1
1 minus 1 A times x bar equals square root
of 2 the Eigen value square root of 2 times
x bar this is from the definition of Eigen
value this implies. ah Now, you can which
is also basically square root of 2, now you
can insert an identity identity times x bar.
Which implies now you bring it this on the
left. So, this will become a minus square
root of 2 times identity which is 1 minus
square root of 2 1 1 minus 1 minus square
root of 2 into x bar, now I will write x bar
as a vector it is a two dimensional vector
x 1 x 2 equal to 0. Now, this implies that
you get equations in x 1 and x 2 1 minus square
root of 2 into x 1 plus x 2 equal to 0 ok
and, x 1 minus 1 plus square root of 2 into
x 2 into 0. Now, this equation if you multiply
by 1 minus square root of 2, you will realize
that you get again the same equation.
So, this will be 1 minus square root of 2
into x 1 minus 1 plus square root of 2 into
1 minus square root of 2 so, this minus of
minus 1 so, that is plus 1 plus x 2 equal
to 0. So, implies that basically the first
and the second equation are identical all
right. So, basically you have just one equation
and therefore, this is an infinite number
of solutions and that is kind of obvious,
because if the eigenvector corresponding to
Eigen value is not unique. So, that is if
k x is k x bar is an eigenvector, then ah
x bar scaled by any constant k is also an
Eigen vector corresponding to the same Eigen
value ok.
And therefore, ah correspond to there are
infinite number of eigenvectors in that sense
ok. So, this means that these two equations
are basically the same.
Now, what we will do is to derive a solution
you set x 1 to derive any solution, or one
such solution you set x 1 equal to 1, this
implies x 2 equals minus of 1 minus root 2
and this you can verify is an eigenvector,
that is we look at x bar equals x 1 x 2 that
is 1 minus 1 minus root 2 that is root 2 minus
1, or not ah minus 1 plus root 2, or root
2 minus 1 and this you can check. This is
an one of the eigenvectors of A ok.
One of the Eigen vectors of matrix and you
can check this ok, let us do that if you look
at A times x bar that is equal to 1 1 1 minus
1 times x bar that is 1 root 2 minus 1. So,
this will be 1 into 1 plus root 2 minus 1
so, this will be root 2 1 into 1 minus root
2 minus 1.
So, this will be 2 minus root 2, which if
you pull out this constant square root of
2, this will be well 1 square root of 2 minus
1. And which is basically nothing, but if
you call this as your x bar this is nothing,
but lambda times x bar where what is lambda
lambda equal square root of 2.
And we already seen x bar is 1 square root
of 2 minus 1 so, this verifies that verifies
that square root of 2 equals the Eigen value
and the fact that 1 comma, it verifies both
right and that 1 comma square root of 2 minus
1 equals an eigenvector ok. So, this verifies
basically both the facts that square root
of 2 is the Eigen value of this matrix A and
square root of ah 1 square root of 2 minus
1 is the eigenvector.
Now, similarly one can find the other eigenvector
all right corresponding to Eigen value minus
square root of 2. Similarly 
similarly for Eigen value minus square root
of 2, we have 1 1 1 minus 1 of x bar equals
minus square root of 2 x bar, this implies
1 1 1 minus 1 plus square root of 2 times,
identity into x bar equals 0.
This implies that basically if you look at
this this implies 1 plus square root of 2
1 1 minus 1 plus square root of 2 into the
vector x bar, into the vector x bar that is
x 1 x 2, this is equal to 0 and this implies
basically 1 plus square root of 2 times x
1 plus x 2 equal to 0 and, you can see both
the equations will reduce to the same thing.
And now once again what we will do is we will
set x 1 equals 1, that implies x 2 equals
minus of 1 plus square root of 2, or minus
1 minus root 2. And therefore, the Eigen vector
x bar equals ah 1 minus 1 minus root 2 ok.
So, this is the other eigenvector corresponding
to Eigen value minus square root of 2.
This is the other eigenvector, corresponding
to the other Eigen value minus square root
of 2 all right. So, that is a brief introduction
to the concept of eigenvectors and Eigen values
of the matrix. Let us look at another important
concept that is the concept of symmetric and
the Hermitian symmetric matrices, these are
Hermitian matrices all right.
So, what we want to look at now is basically
the notion of what is what are known as symmetric
and Hermitian 
symmetric and Hermitian matrices. So, let
us say A is a real matrix n cross n real matrix,
we say symmetric A 
is symmetric, if A equals A transpose that
is this is symmetric equals transpose a transpose
that implies take any element a i j, that
is equal to a j i for all pairs i comma j
ok. So, basically for a symmetric matrix we
must have that equals A transpose. And naturally
that implies this must be a square matrix
all right, because only this is the symmetry
is only preserved of the matrix is a square
matrix.
And it is Hermitian or a Hermitian symmetric
matrix Hermitian, if a equals a Hermitian
and what is a Hermitian. Now, let us say A
is our matrix a 1 1, a 1 2, a 2 1, a 2 2 and
so, on. For the Hermitian matrix what you
have to do is you have to take the transpose
and, you have to take the complex conjugate
of each element. So, this will be a 1 1 conjugate,
since you are taking the transpose this becomes
a 1 a 2 1 and the conjugate of that a 2 1
becomes a 1 2 and the conjugate, this is a
2 2 and the conjugate ok.
So, basically for the Hermitian you take transpose
plus 
conjugate of each element. Now, A equal to
A Hermitian that is it is Hermitian matrix
implies that a i j equals a j i conjugate
that is this is Hermitian symmetric, if a
1 2 equals a 2 1 conjugate and so, on ok.
So, that is that matrix is known as a Hermitian
matrix ok.
So, this basically is a Hermitian matrix ok.
Now, there are several interesting properties
of this Hermitian and symmetric ah matrices,
one of the most interesting properties is
that the Eigen values of both symmetric and
Hermitian matrices are real all right.
So, the first property is that 
the Eigen values of Hermitian and, symmetric
matrices are real, these are real quantities
all right.
And ah second properties is another interesting
property, Eigen vectors corresponding to distinct
Eigen values that is different Eigen values
not the same Eigen value, but distinct Eigen
values are orthogonal and this is an important
property. This implies that ah if V 1 bar
comma V 2 bar are the eigenvectors corresponding
to distinct Eigen values 
lambda 1, comma lambda 2 this implies for
a symmetric matrix, V 1 bar Hermitian V 2
bar equal to 0. This is the meaning of vectors
being orthogonal that is two vectors are orthogonal,
if that is x 1 bar x 2 bar are two vectors,
they are real vectors x 1 bar transpose x
2 bar is 0, they are complex vectors x 1 bar
Hermitian x 2 bar equal equals 0, then the
vectors are said to be orthogonal ok.
So, this is orthogonality of vectors, this
is an important property in general orthogonality
of vectors is also a very important property
ok. Now, let us go back to our earlier example
to illustrate this fact for instance, you
have probably realized that if you look at
our previous matrix that is 1 1 1 minus 1,
this is a symmetric matrix you can see, this
is a 
symmetric matrix, we have A equals A transpose.
And the Eigen values are natural if you look
at the Eigen values equals plus, or minus
square root of 2 and these are real quantities
ok, we already seen this all right.
So, you can see that the eigenvalues of this
symmetric matrix are real as given by the
property. And, now let us look at the eigenvectors
and we will show that the eigenvectors are
orthogonal the eigenvectors are 1 root 2 minus
1. Let us call this as your V 1 bar and the
other eigenvector is 1 ah minus 1 minus root
2. Now, since these vectors are real we can
simply take the transpose so, this is V 2
bar n because transpose, or Hermitian will
give the same thing for real vectors.
Now, if you look at V 1 bar into V 2 bar we
will bar transpose into V 2 bar that gives
1 square root of 2 minus 1 times 1 minus 1
minus root 2, which is equal to you can clearly
see 1 minus root 2 plus 1 into root 2 minus
1 that is 1 minus 2 minus 1, which is basically
0. So, this basically is showing you that
V 1 bar Hermitian V 2 bar equal to 0 implies,
these vectors are orthogonal ok all right.
That is a very interesting property and that
is arising, because the matrix is symmetric
all right. So, in this module we have looked
at various important and very interesting
and also very important concepts of Eigen
values eigenvectors and symmetric matrices.
And these are very important, because these
are going to be used frequently in our discussion
and the development of the framework of optimization
ah for various applications.
Thank you.
