Hello friends, so, welcome to the 22nd lecture
of this course. So, in the last lecture, we
have learnt few results about normal matrices.
In this lecture, we will discuss another important
class of matrices those are called Positive
Definite Matrices. So, as we know that if
we are having a symmetric matrix, then all
the eigenvalues are real, that is true, and
we have seen the proof of this in the previous
unit.
However, what is additional we will learn
in this lecture that the eigenvalues are not
only the real, but positive also or just non
negative.
So, a square matrix A is called positive definite
if it is symmetric, and for a non 0 vector
x, x transpose into A into x, which is a scalar
and x would be positive. So, whatever non
0 vector you take x, x transpose A into x
would come out to be positive. If this is
true, then we will say the square matrix A
is a positive definite matrix.
In short I am saying it PD P for positive
and D for definite. The same definition in
other words can be written like this; x transpose
into Ax greater than equals to 0 for all x,
and this particular expression x transpose
A into x will be 0, only if x equals to 0.
So, this is the definition of positive definite
matrix. A square matrix A is positive semi
definite or PSD in short, if it is symmetric,
and x transpose Ax is greater than equals
to 0 for all non 0 x. So, the only difference
is positive definite and positive semi definite
is, here it will be strictly greater than
0, for non 0 x here it may be 0 also. The
example or I will say that trivial example
of a positive definite matrix is identity
matrix.
So, that a symmetric matrix A whose eigenvalues
are positive or non-negative is called positive
definite. So, here what we need to prove?
We are having a matrix A, let us say A n by
n matrix, and we have to prove that if all
the eigenvalues of A are positive then it
is a positive definite matrix, and if all
the eigenvalues of A are non-negative, then
it is a positive semi definite matrix. So,
let us take an eigenvalue lambda, let lambda
be an eigenvalue of A 
and x be the corresponding eigenvector.
So, here lambda is eigenvalue and corresponding
eigenvector is x, it means A into x equals
to lambda in 2 x. Now, multiply both side
by x transpose. So, x transpose A into x will
become lambda x transpose into x. Or I can
write it lambda equals to x transpose Ax upon
x transpose x. Now if lambda is positive,
this means x transpose Ax upon x transpose
x is positive.
Since denominator will be always positive,
so, it means x transpose Ax is positive for
all x belongs to; this means A is a positive
definite matrix. Similarly, if lambda is greater
than equals to 0, this means x transpose Ax
is greater than equals to 0 for all x belongs
to Rn. And please note here x is eigenvector
so, they will be non 0 vectors. So, in this
way we can prove that if all the eigenvalues
of a matrix are positive, then the matrix
is positive definite. If all the eigenvalues
of a matrix are non-negative, then the matrix
is positive semi definite.
Alternate definition for positive definiteness
can be given like this. A real symmetric matrix
A is positive semi definite or I will say
a positive semi definite, if and only if A
can be factored as A equals to B transpose
into B. Further A is positive definite if
B is nonsingular.
So, the proof can be seen like this so, A
is positive definite or positive semi definite,
if and only if A can be written as B transpose
into B. So, since A is a symmetric matrix,
we can always find a matrix P such that A
equals to P into D into P transpose. Means,
since A is symmetric so, it will be always
diagonalizable. Now what I need to prove that
A can written as the factor of two matrices
that is B transpose into B. Now this can be
written as P since A is positive semi definite.
So, eigenvalues of a will be non-negative,
and hence all the diagonal entries of D will
be non-negative. So, I can write this D as
D half into D half; where the entries are
square root of the original D into P transpose.
So, if I choose B as D half into P transpose,
then from here I can write A equals to B transpose
into B. If all the eigenvalues are means if
B is nonsingular, then all the diagonal entries
of B will be strictly positive. And hence
A is positive definite in this case so, in
this way we can prove this result. There are
few more test for check in whether a given
matrix is positive definite or not.
And one of the test is based on the leading
principal minors. So, a real symmetric matrix
is a positive definite if the leading principal
minors of A are positive, or all principal
minors of A are positive here positive. Means,
having the positive determinates.
Let us check all these results for a given
matrix. So, check the following matrix for
positive definiteness. The matrix is 2 minus
1 0 minus 1 2 minus 1 0 minus 1 2. So, the
matrix A is given which is given as 2 minus
1 0 minus 1 2 minus 1 0 minus 1 2.
So, we have to check whether this matrix is
positive definite or not. So, there are different
test so, let us take the first definition,
let x belongs to R 3 be a nonzero vector.
So, means x equals to x 1 x 2 x 3 transpose.
So now, x transpose A x will become x 1 x
2 x 3 into 2 minus 1 0 minus 1 2 minus 1 0
minus 1 2, into x 1 x 2 x 3. This comes out
to be 2 x 1 square minus 2 x 1 x 2 plus 2
x 2 square minus 2 x 2 x 3 plus 2 x 3 square.
This I can write like this, x 1 so, minus
x 1 x 2 so, minus half, x 2 whole square so,
if I express it will become x 1 square 2 x
1 square. So, x 1 square is left out here,
and then 3 by 2 x 2 minus 2 by 3 x 3 whole
square plus 4 by 3 x 3 whole square. So, the
above thing can be written in this way, and
you know that the final expression is the
sum of square terms.
So, this will be always positive for nonzero
x. Hence A is positive definite because x
T A x is always positive for non-zero vector
x, this is one of the way of checking the
positive definiteness.
Another way is by calculating the eigenvalues
of the matrix A. So, if we check the eigenvalues
of this matrix A which is given like this,
the eigenvalues comes out to be lambda equals
to 2, 2 plus minus root 2.
So, eigenvalues are lambda equals to 2, lambda
equals to 2 plus root 2 and lambda equals
to 2 minus root 2. If we check it is positive,
this is also positive, and this is also positive.
So, all the eigenvalues of A are positive
this implies that A is positive definite.
Another test is by checking the minor principal
minor. So, matrix is 2 minus 1 0, minus 1
2 minus 1, and then 0 minus 1 2. So, let us
take first this minor so, A 1 which is 2 which
is greater than 0. Now check the 2 by 2 minor.
So, it will be this one so, this is the determinate
of 2 minus 1 minus 1 2. This comes out to
be 4 minus 1, which is 3 greater than 0. Now
check the determinate of A, that is that 3
by 3 minor minus 1 2 minus 1 0 minus 1 2.
So, if I check this determinate this will
be 2, 4 minus 1 minus minus plus 1 minus 2
plus 0, this comes out to be 6 minus 2, equals
to 4 which is also 0. So, all the principal
minors are positive and hence the matrix A
is positive definite. So, in this way I have
told you 3 different methods to check whether
a given matrix is positive definite or not.
What are those? The first one just take a
non-zero vector x and find out x transpose
x, if you can write that thing as a sum of
perfect squares, then the given matrix is
positive definite. Number 2 find out the eigenvalues,
if all the eigenvalues are positive then the
given matrix is positive definite.
If all the eigenvalues are non-negative, then
the matrix is positive semi definite, means
some are positive and some are 0. The third
is minor test, if all the principal minors
are positive then the matrix is positive definite.
In the same manner we can define the negative
definite matrix, a square matrix A is negative
definite in short I will say it N D, if it
is symmetric and x transpose A x is less than
0 for all x, means, for all non-zero x. The
equivalent conditions are A has only non-negative
eigenvalues, the determinants of A’s principal
sub-matrices are negative or A has negative
pivots, these are the equivalent conditions.
Like the test we have seen for the previous
example just the same test, but in negative
sense. To prove a given matrix is negative
definite. In the same way we can define the
negative semi definite in that symmetric matrix
is negative semi definite, if x transpose
into x is less than equals to 0, for all x
those are not 0.
Then the matrix is called negative semi definite,
means, if a matrix are having the eigenvalue
0 and negative, then the matrix is negative
semi definite.
Now, another important definition related
to definite positive and negative definiteness
is quadratic forms. So, for a vector X which
is a real vectors in having n components and
a matrix A which is a real matrix of size
n by n, the scalar function defined by f X
which is X transpose A X, and in summation
form if the entries of A is denoted by a i
j then it can be written as i equals to 1
to n, j equals to 1 to n a i j x i x j is
called a quadratic form.
Further a quadratic form is said to be positive
definite, positive semi definite, whenever
the matrix A is positive definite or positive
semi definite, this is the real quadratic
form. In the similar manner we can define
the complex quadratic form. So, let X be a
complex vector having n components, and A
is n by n matrix having complex entries and
A is Hermitian. Then the expression f X X
star A X which is comes out to be a i j x
i x j conjugate is called a complex quadratic
form.
For example, if a matrix is given to you and
someone ask find out the quadratic form of
this matrix.
So, let A is given to you which is 3 minus
1 2 minus 1 2 0 and then 2 0 1, find the quadratic
form of; let 
x 1 x 2 x 3 be a vector in r 3, then the quadratic
form of A is defined as x transpose A x which
is x 1 x 2 x 3 multiplied with matrix 3 minus
1 2 minus 1 2 0 2 0 1 into x, x is x 1 x 2
x 3. So, this comes out to be 3 x 1 square
plus 2 x 2 square plus x 3 square, because
these square terms will be just multiplied
with the diagonal elements.
Now, the diagonal element of first row will
be multiple of coefficient of x 1 square,
the diagonal entries of second row will be
the coefficient of x 2 square, the diagonal
entries of third row will be the coefficient
of x 3 square. Minus so, minus x 1 x 2 will
come from here, and x 2 x 1 will come from
here. So, minus 2 x 1 x 2, similarly 2 x 1
x 3 will come from here 2 x 3 x 1 will come
here.
So, it will become x 1 x 3 2 plus 2 so, it
will be 4 so, 4. The coefficient of x 2 x
3 will be 0 as well as for the coefficient
of x 3 x 2. So, this is the required quadratic
form of the given matrix. If someone ask to
check whether this quadratic form is positive
definite or positive semi definite or negative
definite or negative semi definite what you
have to check you just check this matrix A.
If A is positive definite the quadratic form
is positive definite. If A is negative definite
the quadratic form is negative definite and
similarly for positive semi definite and negative
semi definite. Well, in this lecture we have
learnt the definition of positive definite
matrices, positive semi definite matrices,
negative definite matrices and negative semi
definite matrices.
Later on we have learnt few test how to check
whether a given matrix is positive definite
or not. Apart from that we have learnt quadratic
form relating to a matrix. In the next lecture,
we will learn some other important properties
of quadratic form. We will learn the diagonalization
of the quadratic form; we will learn some
applications of the quadratic form, in particular
to draw a given quadratic curve with the help
of quadratic form. With this I will end this
lecture.
Thank you very much.
