In the last lecture, we saw, that if A is
in C n n, that is, if A is n by n complex
matrix, then for A to be diagonalizable, for
A to be diagonalizable, we need, that the
algebraic multiplicity is equal to the geometric
multiplicity for every Eigen value of A. What
we mean is, that is, if lambda j is an Eigen
value of A with algebraic multiplicity a j,
then we must have n, we must have a j, linearly
independent eigenvectors corresponding to
lambda j. So, depending on the multiplicity
of the Eigen value we must have that many
linearly independent eigenvectors.
This is the same as saying, that the dimension
of the eigenspace W j corresponding to lambda
j must be, must be a j, must be a j. So, we
need this condition for diagonalizablity.
And we have seen that there are matrices,
there are matrices for which this condition
is not satisfied, this condition is not satisfied,
and hence not diagonalizable. So, there are
matrices for which the condition is not satisfied,
that is, the geometric multiplicity will become
less than the algebraic multiplicity for some
Eigen values and hence, the matrix fails to
be diagonalizable. Therefore, we have this
problem, that given a matrix A, you, priory
we do not know whether it is going to be diagonalizable
or not. We have to look at the geometric multiplicity
and the algebraic multiplicity, that is, we
look whether we get enough numbers of eigenvectors
to form a basis for the whole space.
Now, we are going to look at the class of
matrices. We shall now look at the class of
matrices, which is the sub-class of the entire
set of matrices for which always a m equal
to g m for each Eigen values. So, we are going
to look at matrices, a class of matrices.
Among this whole world of matrices, there
is a class of matrices for which this condition
is always satisfied and hence, diagonalizable.
This is the first sub-class of matrices we
look at. What do you mean by following? We
have this whole collection of n by n matrices,
inside that we are going to look at the sub-class
H n. We will define what H n is and this sub-class
H n.
You take any matrix A in the sub-class in
H n, it will be diagonalizable or for which
algebraic multiplicity, it will be equal to
geometric multiplicity. So, we will call this
sub-class H n, we will explain, what H n is,
in short.
So, for that we will look at some preliminary
ideas, simple calculations, which will give
us the right notational frame work to work
with. So, let us, look at a matrix A, which
is n by n complex. Then, let us denote it
as the entries, as a jk, where j is the row
index, which goes from 1 to n and k is the
column index, which goes from 1 to n. So,
we have a complex matrix A, whose entries
are a jk; a jk denotes the entry in the jth
row and the kth column and it is a complex
number.
Now, take any vector u in C n, and then u
is of the form, u equal to u 1, u 2, u n,
where the u j are all complex numbers. Now,
if u is a vector in C n, then Au, if you multiply
the vector by u the matrix A, that is also
a vector in C n, because A is n by n and u
is n by 1, so the product is going to be n
by 1; it is also going to be vector in C n.
If, you take a vector in C n, Au is also going
to be in C n. So, we can write Au, as it is
1st component, we will denote by Au 1, 2nd
component by Au 2 and so on, Au n.
Now, how do we get the jth component? A, look
at the vector Au, A is untrained by multiplying
them matrix a 11, a 1n, a 21, a 2n and so
on, a n1, a nn with the vector u 1, u 2 and
u n. In order to get the jth component of
this product, we have to look at jth row and
multiply it with the vector u 1, u 2, u n.
So, we get the jth component of Au to be a
j1 u 1 plus a j2 u 2 plus extra, a jn u n,
which we will write in summation notation
as summation k equal to 1 to n, a jk u j.
So, the j, the component of the product A
u, it was given by a jk u j.
Now, consider two vectors x, y epsilon C n.
x is x 1, x 2, x n; y is y 1, y 2, y n. Now,
applying the above logic we get A x is the
matrix and it is component, jth component.
By the above calculation, in this we replace
u by x, we get, that is equal to k equal to
1 to n, a j k x j. That is our A xj.
If you look at the inner product of A (x,
y) by definition, that is summation j equal
to 1 to n, the jth component of x multiplied
by the jth component of y with the conjugate
because we are dealing with the complex vector
space. Now, A xj we have calculated here and
if we substitute that, that becomes j equal
to 1 to n summation k equal to 1 to n, a j
k x k times y j y.
Now, we have two sums, one is on the index
j and the other is on the index k and both
are finite sums, and therefore, with an interchange
in order of the sum. So, we will take the
k sum first and then x, this should be x k,
the x k comes out, the remaining all are dependent
on j, so they all go inside as a j k y j y.
We can now write this as summation k equal
to 1 to n, x k, j equal to 1 to n a jk bar
y j, this quantity bar. We have taken the
conjugate twice and for simple notation we
write this as summation j equal to 1 to, summation
j equal to 1 to n A star k j y j bar.
Where A star alpha beta is a beta alpha conjugate,
from, from the above definition, for any alpha
beta between 1 and n. Therefore, this becomes,
if you now, I define the matrix A star to
be the matrix, whose entries are A star alpha
beta, which is equal to A beta alpha, then
this becomes k equal to 1 to n x k A star
y k y, which is the same thing as the inner
product between x and A star. This is explicatively
seen, this competition of A x, y equal to
x, A star y.
Therefore, what is the conclusion? The conclusion
is that if A belongs to C n by n, we define
A star. How do we obtain as A star? We interchange
the row index and the column index and then
conjugate it. Interchanging the row index
and the column index be among s to transposing
the matrix, so it is A transpose conjugate.
So, A star alpha beta is equal to a beta alpha
bar. If we now define A star as A transpose
bar, then A x, y is equal to x, A star y for
every x, y in C n. This is a very important
identity, which will be used repeatedly.
In particular, in particular if we take everything
real, if A belongs to R n, then there is no
more conjugation involved. So, A star will
be defined as A transpose only. Then, then
we have A (x, y) is equal to x comma, in place
A star we have a transpose A transpose to
y for every x, y. Now, in R n this two are
important observations. For A matrix, very
important identity A (x, y) equal to x, A
star y, that is, if you move A in the inner
product, from one factor to another factor
it moves as a star. Here, A was in the first
factor, now we wanted it to move to the second
factor, it moved as a star. Similarly, if
has comes back from second to here, will come
back again with the star note, that A star
is equal to A.
If, the matrix again, transpose once you transpose
a conjugate again, you transpose a conjugate
you get. So, these two identities are going
to be very useful identities for us. So, let
us look at some examples. Let us take a very
simple matrix, which is 1 plus i, i, 2, 3,
which is now in C 2 2. So, it is a 2 by 2
matrix, is a complex matrix.
Now, what is A star? In this case we have
to transpose and conjugate. So, this conjugation
will make this i, transposition will bring
i here and conjugate will make it minus i
and 2 was there and 3 was here. This is what
A star is,. sSuppose, x is a vector, x 1,
x 2, which is in C 2, then what is A x? A
x is in 1 plus i, i, 2, 3 into x 1, x 2, which
is 1 plus i into x 1 plus i into x 2, 2x 1
plus 3x 2. This is what A x is. So, now, if
we take the inner product of x with y, the
inner product is taken by taking the components
of the product of the components with the
second one coming as a conjugate.
So, the first component of A x times the first
component of y conjugated plus the second
component of A x times the second component
of y conjugated. We will make a slight rejudgement
of this, we will write this as x 1, collect
all the x 1 terms, which is 1 plus i into
y 1 bar and x 2 plus and x 1 comes from 2y
2 bar, then we have plus x 2 into i y 1 bar
plus 3y 2 bar. This is what A (x, y) is.
Let us compute A star y. A star y equal, A
star is the matrix 1 minus i, 2, minus i,
3, so 1 minus i, 2, minus i, 3. Let us look
at this 1 minus i, 2 and minus i, 3 is A star;
y is y 1, y 2. So, if we now take this product,
this becomes 1 minus i into y 1 plus 2y 2,
minus i into y 1 plus 3y 2. And therefore,
x, A star y, that is the inner product of
x with A star y will be the first component
of x 1 times the conjugate of the first component
of A star y, which is 1 plus i into y 1 bar
plus 2 into y 2 bar. Similarly, second component
of x into the conjugate of the second component
of y. Now, compare it with A (x, y), which
we got here and we see, that it is the same
as, thus A (x, y) is equal to x, A star y
for every x, y in C. This is the identity
that we have been discussing above.
Let us look at another example. Let us take
A to be 1, i, 2, minus i. Now, in this case
what is A transpose? It is 1, i, minus i,
2 and A star is the conjugate of the transpose.
So, it is 1, i, minus i, 2. We have to conjugate
the A transpose to get A star. So, again,
what is A x? A x is 1, i, minus i, 2, that
is, A, x is x 1, x 2. If we now take a product,
I get x 1 plus i x 2 minus i x 1 plus 2 x
2. Now, if I take A (x, y), which is inner
product of A x with y, I have to take the
first component of A x multiplied with the
conjugate of the first component of y plus
second component of A x multiplied by the
conjugate of the second component of y.
When we do this product we rearrange these
terms again as before. Collect the x 1 terms,
I get x 1 into y 1 bar minus i y 2 bar plus
x 2 into i y 1 bar plus 2 y 2 bar. Now let
us, compute x, A star y. First of all, what
is A star y? A star y is same as A y because
we observe here, that A star is equal to A,
A star is equal to, is same as A y.
So, it is again 1 i minus i 2 into y 1 y 2,
which is y 1 plus i y 2 minus i y 1 plus 2
y 2. Now, therefore, if i take (x, A star
y), the inner product of x with A star y,
I am x 1 times the first component of, we
have to put the first component of A star
y with the conjugate, so it will be y 1 minus
i y 2 y 1 bar minus i y 2 bar plus x 2 into
the second component of A star y with conjugation
i y 1 bar plus 2 y 2 bar, which is precisely
what we got here for (A x, y) and. Therefore,
(A x, y) is equal to x, A star y. In this
case, A star is…
Let us look at another example, the real case.
Consider the matrix 1, 2, 3, 4, what is A
transpose? That is 1, 2, 3, 4, Ax is x 1 plus
2x 2, 3x 1 plus 4x 2, A transpose y is x y
1 plus 3y 2, 2y 1 plus 4y 2 and therefore,
(A x, y). Now, we know, we do not have any
conjugation because now we are looking for
x, y in R n.
So, if you now take x, y in R n, all real,
then (Ax, y) is first component of Ax into
the first component of y plus the second component
of Ax into the second component of y, which
we will rearrange again as before, x 1 into
y 1 plus 3y 2 plus x 2 into 2y 1 plus 4y 2.
On the other hand, we have x comma A transpose
y is equal to x 1 into the first component
of A transpose y, which is y 1 plus 3 y 2
plus the second component of x into the second
component of A transpose y. You, you compare
these two and be sure, that these two are
equal and therefore, (Ax, y) is equal to x
comma A transpose y for every x y in R n.
So when we have, so let us again summarize
with the examples and… So, first A belongs
to C n n, we define A star to be transpose
the matrix and then conjugate and then (Ax,
y) is equal to x comma A star y for every
x, y. A star is called the Hermitian conjugate,
is called the Hermitian conjugate of…
Then, the second thing is, we observe is A
is in R n, then (Ax, y) is equal to x comma
A transpose y for every x, y in R n. So, now,
we observed, that when A star is equal to
A in this example two in the above, particularly
if I look at this identity when A star is
equal to A, then we get A x equal to (x, Ay),
that is, we can freely move A from one factor
to the other factor without any change. If
A star equal to A, when we move this A to
be second factor, it will still move as A
1 and that makes things work much nice.
We now make a special name for such matrixes,
so we now introduce the notion of a Hermitian
matrix. A matrix A, which is complex and n
by n is set to be Hermitian if A star equal
to A. So, the conjugate Hermitian conjugate
is itself, so it is self conjugate matrix;
so, it is self conjugate matrix in the sense
of Hermitian conjugation, the Hermitian conjugation
transpose conjugate. If you transpose the
conjugate, the matrix, if you get back the
original matrix, then it is called a Hermitian
matrix.
For example, if A equal to 1, i, i, 1; A transpose
is 1, i, i, 1. Therefore, A star is equal
to A transpose conjugate is 1, minus i, minus
i, 1 and this is not equal to A.
And therefore, A is not Hermitian. On the
other hand, look at this example, A equal
to 1, i, minus i, 1, then A transpose is 1
minus i, i, 1, rows are written as columns
and columns as rows. Therefore, A star, which
is the conjugate of A transpose is 1, i, minus
i, 1, which is equal to A and therefore, A
is Hermitian.
Therefore, A 
is Hermitan, if and only if its star is itself.
In particular, when we are dealing with real
matrix is conjugation, is no effect, A star
means, is same as transpose.
If A belongs to R n and A transpose equal
to A, we say, A is a real symmetric matrix.
Note, that a real symmetric matrix can be
thought of, is the complex Hermitian matrix
because the real numbers can be thought of
as complex. So, starring again does not affect,
A transpose conjugate will still be A. So,
note, a real symmetric matrix can be thought
of also as a complex Hermitian matrix.
Now, suppose, A is Hermitian and we denote,
we, we say, A is Hermitian if A star is A.
Therefore, if you look at the diagonal, what
does that mean? This mean a jk bar let us,
using the following notation, correct notation…
So, the, the jkth entry of the starred matrix
is obtained by the kjth entry of the original
matrix with conjugate. In particular, if j
equal to k, we get the diagonal entries. We
get the jth for, for the jth diagonal entry,
a jj star must be equal to a jj bar. Now,
if A is Hermitian, a jj star is the same as
a jj because A star is equal to A and therefore,
a jj bar is equal to a jj, which says a jj
is real.
Therefore, for a Hermitian matrix, all the
diagonal entries must be real. The diagonal
entries of a Hermitian matrix, Hermitan matrix,
should all be real. The matrix may be complex,
but when the matrix has to be Hermitian conjugate,
the diagonal entries are forced to be real
numbers. So, you cannot have a complex Hermitian
matrix with complex diagonal entries, all
the diagonal entries must be real.
Now, we shall denote by H n, the set of all
n by n complex Hermitian matrices. So what
is H n? H n is all those matrices in C n n,
the complex Hermitian complex matrix is n
by n such that A star is equal to A. So, this
is the collection of all Hermitian matrices.
Now, it is this class of matrices, which are
having a very nice set of properties as for
the eigen values in eigen vectors are concerned
and it is this class, that it will be very
useful in all our computations and answering
many of your questions, that we rise in the
beginning of the course.
So, we shall study this class a little bit
more closely. It is first, some simple properties
of H n of this collection. Look at some simple
properties of this collection of matrices.
First, we have observed, that the moment it
is Hermitian, so if A belongs to H n, (Ax,
y) must be equal to (x, Ay) for every x, y
in C n, because A star is equal to A. We had
(Ax, y) is equal to x comma A star y. But
since, A star is equal to (Ax, y) must be
equal to x comma A star. So, this is the first
property, which every matrices in H n possess,
that is, in an inner product the factor A
can be moved from the first to the second
without any change.
The second is, as we have observed above,
if A belongs to H n, then all diagonal entries
of A must be real; all diagonal entries of
A must be real.
Let us now look at the property 1 as in a
special situation. So, in 1, if we put x equal
to y, we get (Ax, x) is equal to (x, Ax) y
is equal to x. So, (Ax, y) becomes (Ax, x)
and (x, Ay) becomes (x, Ax). So, we have in,
we have this simple thing when could they,
so this is true for every x in C n. y is also
taken to be equal to x, so that becomes for
every x in C n. Now, but the right hand side,
by the inner product, inner product of a vector
with itself the conjugate, when the order
is reversed, so we have got (Ax, x). So, therefore,
(Ax, x) is equal to Ax comma x bar.
A number is equal to its own conjugate means,
that number must be real. So, that says, (Ax,
x) is real for all x in Cn, so this is the
third important property. Not only the diagonal
entries are real, the many things are going
to be real for a Hermitian matrix; not only
the diagonal entries are real.
We now see, that (Ax, x) is real for all x
in C n, whatever x, the x may be complex a,
A is a complex matrix, only thing we know,
it is complex Hermitian matrix. So, there
are many non-diagonal entries, which are complex,
x could be highly complex matrix vector and
x, if A is a Hermitian, (Ax, x) must be real.
All the complexity is gone and everything
becomes real. So, (Ax, x) is real for all
x in R n. It is a very, very important property,
so which we will be using repeatedly.
So, we have seen now three properties, this
is the first fundamental identity for Hermitian
matrices (Ax, y) equal to (x, Ay) for all
x, y in C n. All diagonal entries must be
real and (Ax, x) must be real for all x in
C n.
Suppose, A and B are, A and B are Hermitan
matrices, suppose A and B are Hermitian matrices,
that is, A star equal to A and B star is equal
to B, both are Hermitian matrices. Suppose,
we take two Hermitian matrices and we look
at their sum, call that as C. Let C be equal
to A plus B, the sum of these two matrices.
Then, C transpose is A transpose plus B transpose,
because the transpose of a sum is the sum
of the transpose, thus was C transpose conjugate
is A transpose conjugate plus B transpose
conjugate. This is C star and this is A star,
this is plus B star, that is, the sum of the
star is star of the sum. So, C star is equal
to A star plus B star. In particular, if A
and B are Hermitian, this is the same as A
plus B because A and B are Hermitians.
So, if A and B Hermitian, A star is equal
to A and B star is equal to B, but A plus
B was C, that means, C star is equal to C,
that means, C is also Hermitial. So, conclusion
is A, B are Hermitian, implies their sum is
also Hermitian. This sum of Hermitian matrices
is a Hermitian matrix.
However, there is certain, that is the 4th
important property. However, there is a slight
problem as far as com product, scalar multiple
on products are concerned. Let us look at
a Hermitian matrix and take any complex number
C. Then, let us define C to be alpha times
A, that is, the matrix A is multiplied by
alpha, which means, every entry is multiplied
by alpha. So, C transpose is alpha times A
transpose and therefore, C transpose conjugate
is alpha conjugate A transpose conjugate,
that is, alpha conjugate A star. Now, therefore,
C star is equal to alpha conjugate A star.
Therefore, C is Hermitian if and only if C
star is equal to C, if and only if alpha bar
A star, that is, the C star must be equal
to alpha A, if and only if alpha bar A is
equal to alpha A, because A star is A. We
have assumed that A is in H n. Since A is
in H n, A star can be replaced by A. So, C
will become Hermitian, if and only if alpha
bar A equal to alpha A. Now, this is satisfied
if A is zero matrix. If A is not the zero
matrix, then alpha must be equal to alpha
bar, if and only if A equal to 0 n cross n
or alpha is real and therefore, if we have
a non-zero Hermitian matrix, its scalar multiple
is also Hermitian if and only if the scalar
is real.
So, therefore, A belongs to H n, A not equal
to 0 n. then, implies alpha A is Hermitian,
is also in H n if and only if alpha is real.
So, this is the scalar multiple of a Hermitian
matrix, will become Hermitian only if the
scalar, which is multiplying is real. This
is same thing as saying H n.
The class of all Hermitian matrices is a vector
space, not over the field of complex numbers,
but over R because addition of two Hermitian
matrices is Hermitian. So, addition no problem,
scalar multiple in order, that it be close
with respect to scalar multiple, we have to
take only scalars to be real. That is the
problem with R, the constraint with respect
to scalar multiplication.
The next property is look at the product of
two. This is the product. Suppose A is a Hermitian
matrix and B is also a Hermitian matrix. Let
us define C to be the product, define C to
be the product AB. So, we have two Hermitian
matrices, we are looking at their product.
What is C transpose? It is AB transpose, but
AB transpose, the transpose of the product
is the product of this transpose in the reverse
order and therefore, C transpose conjugate
is B transpose conjugate into a transpose
conjugate.
And this is C star and this is B star and
that is A star and that is equal to BA, because
B and A and B are Hermitian. Since A and B
are Hermitian, B star is, B and A star is
A. Now, therefore C star is equal to C, that
is, C will be Hermitian if and only if C star
is BA, C is AB, that is, if and only if A
and B are commutable. Therefore, that is the
next property.
Product of two Hermitian n by n matrixes is
an n by n Hermitian matrix, Hermitian matrix,
if and only if the two matrixes commute. These
are some of the important properties of a,
herm, the collection of Hermitian matrix.
Let us go over them.
The first property we have was, that we must
have (Ax, y) equal to (x, Ay) for every x,
y in C n. Then, we must have, that all the
diagonal entries must be real. Then, (Ax,
x) is always real, that is the 3rd property
and the 4th property is that the product,
the sum of the two Hermitial matrixes is Hermitian
always and should be the 5th property, the
number in this problem. The 5th property is
that A is Hermitian, then the scalar multiple
is again Hermitian, if and only if all the
scalar is real. And this is the 6th property
is about the product. The 4th and the 5th
properties together give us, that H n is a
vector space over R. It is not a vector space
over C; is not a vector space over C. Then,
the product of two Hermitian matrixes is again
Hermitian if and only if the two matrices
is commute.
So, now, we have this fundamental properties
of Hermitian matrices and we again stress
the two of the most important properties,
which will repeatedly use is the fact, that
(Ax, y) equal to (x, Ay) for all x, y and
(Ax, x) is real for all x. This is a, these
all are two characteristic properties of Hermitian
matrixes (Ax, y) equal to (x, Ay) for all
x, y and (Ax, x) is real.
So, now, we are going to look at this class
of matrices, which is closed and under addition,
which is closed and real scalar multiplication,
which is not closed under multiplication because
the product of the two Hermitian matrices
need not be a Hermitian matrices. The product
becomes Hermitian if and only if the two matrices
commute with each other. Commutatively is
an important property of matrices, which has
a lot of things to say about between the two
matrices what happens.
The class H n of Hermitian matrices 
exhibit nice Eigen properties. That is the
reason why we look at this and deal with these
matrices so very often because as for this
Eigen properties, Eigen values, Eigen vector
properties and their structure, they are very
nicely built-in, which makes them automatically
diagonalizable, not only diagonalizable, but
nicely diagonalizable.
So, in this context, before we get to study
the Eigen properties of these matrices, we
shall introduce certain notations and terminologies.
Let us say, U is a matrix whose columns are
phi 1, phi 2, phi n. So, U is a matrix, therefore
phi j belongs to C n. Each column is n component
vector. So, we have a matrix U whose columns
are phi 1, phi 2, phi n and suppose, what
does this mean? This means, that phi j, phi
k are orthogonal to each other and each vector
has length one, which means phi j are ortho-normal
vector, that is, phi 1, phi 2, phi n are ortho-normal
vectors. So, a matrix U in which the column
forms ortho-normal set of vectors.
Then, we have U star is transpose conjugate,
so it will be phi 1 star, phi 2 star, phi
n star and when we multiply U star and U we
get phi 1 star, phi 2 star, phi n star, that
is, U star into U is phi 1, phi 2, phi n.
We now multiply, first we get phi 1 star,
phi 1, which is 1 because phi 1 star, phi
1 is the inner product of phi 1 with phi 1.
So, that is 1, phi 1 star, phi 2 is 0 because
phi 1 and phi 2 are orthogonal and we go on
getting 0. And in the second row we get phi
2 star, phi 1, which is 0 again because the
inner product within phi 2 and phi 1 is 0,
phi 2 star, phi 2 is 1, again we get this.
So, we see, that we get the matrix I n by
n, which means U star is the inverse of U
and y.
Therefore, U star U equal to identity I n
by n, hence U star is equal to U inverse and
U is this U star inverse. Any matrix, which
has this property, is called a unitary matrix.
So, A complex matrix U in C n by n, is said
to be is the definition, is said to be a unitary
matrix.
If U star U equal to I and since U and U star
become inverses of each other, this is the
same as the U star. That is, U star is equal
to U inverse and U star inverse is equal to
U, then we say it is a unitary matrix. We
see, that if the columns are ortho-normal
vectors, and then automatically U is a unitary
matrix. In the real case, we have to replay,
there is no conjugation, so U star is same
as the transpose.
So, we have, we define A, a matrix, let us
call it as O, let us use a different symbol,
let us put this O belonging to R n is said
to be a real orthogonal matrix. If O transpose
O is equal to identity n cross n, that is
O transpose is O inverse and O transpose inverse
is equal to O. So, in the real case, we have
an orthogonal matrix notion and the complex
case, we have the unitary matrix notation.
The commonality is, that if you are in the
unitary case in the complex situation, that
columns form an ortho-normal set of vectors.
In the real case, when we are having an orthogonal
matrix with real in a product, the columns
form an orthogonal matrix. So, now, we are
going to look at the Eigen properties of H
n, this is the most convenient class of matrices
for which the Eigen properties are very nice
and this will form the subject matter for
the next lecture.
