Hello, welcome to another module in this massive
open online course . So, you are looking at
examples to understand the framework or the
mathematical framework or preliminaries required
for ah developing the various optimization
problems . Let us continue our discussion
ah in this module, let us start by looking
at the Eigenvalues of a Hermitian Symmetric
Matrix, ok.
So, we want to look at the Eigenvalue Decomposition
ah or Eigenvalues and Eigenvalue decomposition
of a Hermitian matrix . Hermitian Symmetric
that is if a matrix is Hermitian Symmetric
we know. So, this is our example number 5.
So, recall matrix is Hermitian symmetric if
A equals A Hermitian ok.
So, m cross ah, so for any m cross n. So,
if A remember this is for complex m cross
n matrix and remember Hermitian is nothing,
but you first take the transpose of A and
then take the complex conjugate. So, Hermitian
involves two step, Transpose plus Conjugate.
So, first you take the transpose of the matrix
m cross n matrix becomes an n cross m matrix
and then you take the complex conjugate of
each element and if then you because we have
a matrix which is A equals that is where is
the property that A equals A Hermitian this
is known as a Hermitian symmetric matrix or
simply sometimes the Hermitian matrix ok and
ah naturally for a Hermitian matrix for matrix
to be Hermitian symmetric it must be the case
that it is a square matrix correct. So, that
an n cross n matrix remains an n cross n matrix
when you take its transpose, ok .
So, this implies for a Hermitian matrix , implies
essentially also that matrix is square , makes
sense only for a square matrix . Now we want
to prove some of the properties in this example,
we want to prove some of the properties of
this Hermitian symmetric . Hermitian symmetric
matrices ah that is Eigenvalues of a Hermitian
Matrix are real . The first property is Eigenvalues
of Hermitian matrix are REAL that is if you
look at I consider the Eigenvalues of Hermitian
matrix these are real ok.
So, let to prove this let us consider x bar
be the Eigenvector of A, Lambda equals the
corresponding Eigenvalue , ok. Taking the
Hermitian , now what we are going to do is
we are going to take the Hermitian on the
left and right side. So, we have now, Ax bar
Hermitian equals lambda x bar Hermitian and
you will , you will use the property that
A B Hermitian product of two Hermitian . So,
A B Hermitian that is the product of two matrices
are Hermitian A B Hermitian is B Hermitian
times A Hermitian.
So, this implies ah basically that x bar Hermitian
that is Hermitian vector, Hermitian of vector
x bar times A Hermitian equals x bar Hermitian
times lambda Hermitian, but lambda is a scalar.
So, this, I can simply write as and of course,
this is just a number so, I can simply write
as x bar Hermitian into lambda conjugate since
lambda equals a scalar quantity that is lambda
is simply A, is lambda is simply a number.
So, for a simple number ah the Hermitian of
the quantity is simply taking the complex
conjugate ok and now we have this property
x bar Hermitian A Hermitian equals lambda
conjugate x bar Hermitian.
Now we multiply on left and right by x bar.
So, this implies x bar Hermitian A Hermitian
x bar equals lambda conjugate , ah x bar Hermitian
x bar, but look at this x bar Hermitian x
bar this is norm x bar square. So, this is
equal to norm x bar square lambda conjugate
times norm x bar square and further Ax bar
equals lambda times x bar. So, this implies
x bar Hermitian lambda x bar equals lambda
conjugate norm x bar square and this implies
now lambda is a number.
So, this is lambda x bar Hermitian x bar equals
norm x bar square equals lambda conjugate
norm x bar square and from this from the left
hand side and right hand side since x bar
square, norm x bar square is not equal to
zero this implies lambda equals lambda conjugate
, which basically leads us to the conclusion
that lambda is a Eigenvalue of a Hermitian
symmetric matrix this implies that lambda
equals a real quantity ok.
So, Eigenvalues of, so this implies Eigenvalues
Hermitian 
Symmetric Matrix are real, Eigenvalues of
Hermitian symmetric matrix are real. Now how
about eigenvectors of a Hermitian symmetric
matrix.
Now eigenvectors of Hermitian symmetric matrix
satisfy an interesting property that is eigenvectors
of a Hermitian symmetric matrix corresponding
to distinct Eigenvalues are orthogonal that
is their inner product is zero, we will demonstrate
this fact. So, the property number 2 and another
very interesting property , both these properties
of Hermitian symmetric matrices very interesting
and have immense utility . The second property
is that Eigenvectors , Eigenvectors of, Eigenvectors
of Hermitian symmetric matrix corresponding
to distinct Eigenvalues , corresponding to
distinct Eigenvalues,these are, these are
orthogonal and this is a very important and
interesting property ok.
So, let us consider two Eigenvectors x i bar
comma x j bar, these are two Eigenvectors
and remember these correspond to distinct
Eigenvalues, ok. These are the corresponding
Eigenvalues and remember these are distinct
, lambda i not equal to lambda j that is these
are distinct ok
And ah now what we want to show that the inner
product x i bar Hermitian x j bar equals zero
and the proof can proceed as follows ah that
is if you look at a x i bar equals lambda
i x i bar and ah this implies now if you take
multiply by xj bar Hermitian we have x j bar
hermitian A x i bar equals lambda i or x j
bar Hermitian 
lambda x i bar which is basically lambda x
j bar Hermitian x i bar ok lambda x j bar
Hermitian x j bar .
So, let us call this as the first observation
or the first result x j bar Hermitian Ax i
bar equals x j or Hermitian lambda x i bar
its lambda x j bar Hermitian x i bar . Now
we also have x j bar is an eigenvector corresponding
to the Eigenvalue lambda j which implies A
x j bar equals lambda j x j bar this implies
that ah x j bar Hermitian taking the Hermitian
x j bar or a x j bar, let us just write that
one step A x j bar Hermitian equals lambda
j, x j bar Hermitian this implies x j bar
Hermitian A hermitian equals ah x j bar once
again, x j bar Hermitian, we have already
seen lambda j conjugate because lambda is
once again it is an Eigenvalue its simply
a scalar quantity.
So, this is lambda conjugate , but remember
the Eigenvalues of Hermitian matrix are real
which implies that lambda j conjugate equals
lambda j ok. So, we will use that property
here, this is equal to lambda times x j bar
Hermitian since lambda equals lambda conjugate
and this implies that well again realize that
this is the Hermitian symmetric matrix A Hermitian
is simply A ok.
So, we have x j x j bar Hermitian into a equals
lambda conjugate into x j bar Hermitian . This
implies that now if you multiply by x i ah,
I am sorry this is lambda now if you multiply
by x I bar we have x j bar Hermitian A x I
bar I am sorry this is lambda j this is lambda
j this is lambda j this is equal to ah lambda
j times x j bar Hermitian lambda j x j bar
Hermitian x i bar ok.
So, x j bar Hermitian x i bar equals lambda
j x j bar Hermitian ah x i bar and this we
can , this we can denote as result 2 and now
if you see from result 1 and result to x j
bar Hermitian a x bar x i ah x i bar equals
lambda or lambda. In fact, lambda i , lambda
i x j bar Hermitian x i bar. Similarly if
you look at result 2 x j bar Hermitian A x
i bar equals lambda j x j bar Hermitian x
i bar.
So, this implies from 1 comma 2 from results
, 1 comma 2 what we have , this implies well
we , this implies lambda i x j bar Hermitian
x i bar equals lambda j x j bar Hermitian
x i bar which implies now lambda i minus lambda
j into x j bar Hermitian x i bar equals zero.
And now, since lambda i not remember that
is a key point lambda i is not equal to lambda
j otherwise lambda i minus lambda j can be
zero. So, this implies x j bar Hermitian x
i bar this equal to zero .
So, this finally, verifies the fact that Eigenvalues
correspond Eigenvectors corresponding to distinct
Eigenvalues are basically real ok . So, the
Eigenvalues corresponding to ah, the distinct
Eigenvalues are basically real. Ok let me
just mention this or eigenvectors corresponding
to distinct Eigenvalues are orthogonal that
is what I meant to say . Eigenvalues corresponding
to distinct Eigenvalues are eigenvectors corresponding
to distinct Eigenvalues are orthogonal or
a Hermitian symmetric matrix ok and these
are two important and interesting properties
of Hermitian symmetric matrices that one uses
frequently during ah the development of various
ah, various techniques for optimization, ok
all right.
Let us look at another example,example number
Eigenvalue decomposition of Hermitian matrices
, Eigenvalue decomposition of Hermitian matrices
ah for a Hermitian matrix that is consider
that is n cross n complex Hermitian matrix
equal to A Hermitian that is Hermitian symmetric,
let the Eigenvalues be 
equals or the Eigenvalues and ah V 1 bar V
2 bar V n bar be the corresponding eigenvectors
ok.
Ah now observe that ah let these eigenvectors
be unit norm, ok we can simply normalize remember
an eigenvector if you normalize it, it still
remains A eigenvector because you are simply
dividing it by a ah by it is norm or by a
constant ok that is you are simply scaling
an eigenvector ok. So, let us consider the
Eigenvectors to be unit norm, all right.
So, let norm V 1 bar equals norm V 2 bar equals
norm V n bar,this implies eigenvector is equal
to unit norm and further from the property
previously let us assume that the Eigenvalues
are distinct which implies V i bar Hermitian
V j bar equal to zero for i not equal to j
that is if the Eigenvalues are distinct then
the eigenvector satisfies the property V i
bar Hermitian V j bar equals zero ok.
Now, notice that if you consider this, observe
that if you consider this matrix Vone bar,
V 2 bar, V n bar, the matrix of eigenvectors
ah if you now perform V Hermitian V then what
you are going to have is you are going to
have V 1 bar Hermitian V 2 bar Hermitian V
n bar Hermitian times V 1 bar V 2 bar V n
bar which is equal to.
Well if you look at V 1 bar Hermitian, V 1
that is 1, V 1 bar Hermitian,V 2 is zero V
2 bar Hermitian V 1 is zero V 2 bar Hermitian
V 1 is 1. So, this you can see is simply the
identity matrix which implies V is the inverse
of V Hermitian and V Hermitian is the inverse
of V. So, we have, well we have something
interesting ah, what we have is, we have V
Hermitian V.
So, we have V Hermitian V equals identity
implies V equals V Hermitian inverse and since
the inverse of A square matrix is unique and
this also implies V Hermitian equals V inverse
and this also implies that since if A is B
inverse ah A B is identity be A is also identity
this also implies that V, V Hermitian equals
identity since if A B is identity then B A
is also identity for square matrices ok .
So, we have and such a matrix V which satisfies
V V Hermitian equals V Hermitian V equals
identity such a matrix is termed as a unitary
matrix. So, V is termed as this satisfies
this interesting property that is termed as
a unitary matrix that is V matrix V, ah square
matrix which satisfies this property V V Hermitian
equals V Hermitian V equals identity is said
termed as a unitary matrix, ah unitary matrix
ok.
Now let us look at the product A V A times
matrix V that is A into the eigenvectors V
1 bar , V 2 bar , V n bar , you can see this
is nothing, but this equals well A V 1 bar,
A V 2 bar, A V n bar 
which equals if you look at this, but if these
are eigenvectors so, A V 1 bar is lambda 1
times V 1 bar, AV 2 bar is lambda times V
2 bar A V n bar is lambda n times V n bar
, these are the various columns which you
can now write also as remember we are looking
at a times the matrix V .
So, A times the matrix V equals now you can
write this as V 1 bar, V 2 bar, V n bar times
the diagonal matrix, lambda 1 , lambda 2,
lambda n this is a diagonal matrix.
So, this is nothing, but your matrix V and
this we denote by the matrix capital lambda,
which is the let me just write it with a little
so, this we denote this by the V and this
is basically your capital lambda and what
is this, this is the diagonal matrix of Eigenvalues
,
This is the diagonal matrix of Eigenvalues.
So, we have A times V equals V times lambda
multiplying on both sides by V Hermitian A
times VV Hermitian equals V lambda V Hermitian,
this implies the matrix A which is Hermitian
symmetric can be expressed as V lambda V Hermitian
where V is the matrix of Eigenvectors and
lambda is the diagonal matrix of Eigenvalues.
So, what is V , V equals the matrix of Eigenvectors
lambda equals diagonal , 
equals diagonal matrix of lambda, equals diagonal
matrix , 
lambda equals diagonal matrix of Eigenvalues
and this is termed as the Eigenvalue Decomposition
.
This is termed as the Eigenvalue Decomposition
ok, Eigenvalue Decomposition of A and this
has many interesting properties for instance
if you want to compute the square A into A
which is equal to A square, this will be equals
you can write it in terms of V lambda V Hermitian
into V lambda V Hermitian which is equal to
now V V Hermitian is identity. So, which is
equal to V lambda,lambda V Hermitian equals
V lambda square V Hermitian.
So this is A square ok . Similarly you have
many other interesting properties for instance
you can show , you can generalize this ah
as Araise to n for this Hermitian symmetric
matrix is V lambda n V Hermitian and lambda
to the power of n is easy to compute because
ah since it is a diagonal matrix. So, all
it is it contains ah lambda i each lambda
i raised to the power of n on the diagonal.
So, lambda raised to the power of n this is
very easy to compute, this is simply the matrix
if you think about this, this is lambda 1
to the power of n, lambda 2 to the power of
n . So, on lambda n, I am sorry I should have
used a different integer here lambda m , lambda
ah n raised to the power of m ok.
So, this is simply raised to the power of
m, raised to the power of m,raised to the
power of m. So, this is easy to compute and
frequently you will see this is a very interesting
ah property as well as this is a very handy
tool to perform several matrix,matrix manipulations
that is a Eigenvalue Decomposition of a matrix
all right and it is also one of the fundamental
decompositions of a matrix or one of you can
also call it as one of the fundamental properties
or one of the of a matrix, all right. So,
we will stop here and continue with other
aspects in the subsequent modules.
Thank you very much .
