In the last lecture we discussed about almost
sure convergence. Today we will discuss some
weaker sense of convergence, ah namely mean
square convergence and convergence in probability.
So, we discussed about ah that is strong convergence
, we discussed under that almost sure convergence
.
We will discuss now above some weak convergence
concept weak convergence. There are several
modes of weak convergence . They are like
mean square convergence m.s. convergence,
mean square convergence , 
convergence in probability in probability
and convergence in distribution . We will
discuss 3 mode of convergence, convergence
in distribution . So, out of these 3 we will
ah cover m.s. convergence and convergence
in probability today. These convergence concepts
are called weak convergence, because of the
relaxed relax convergence conditions.
So, first we will discuss about convergent
in mean square sense. We will first give the
definition. A random sequence X n is said
to converge in the mean square sense, this
is abbreviated as m.s. to a random variable
x if E of X n minus X whole square that tends
to 0 as n tends to infinity .
Now, ah this weak convergence concepts are
considered in terms of some deterministic
quantity. Convergence of some deterministic
quantity. In this case, this mean square error
E of X n minus X whole square. This quantity
is a deterministic quantity and we required
at as n tends to infinity this quantity goes
down to 0. Then we say that X n convergence
m.s. sense to X; where X is called a mean
square limit of the sequence and ah it is
written as just like ah in the case of limit
in ah deterministic function, here also we
write l.i.m. of X n as n tends to infinity
is equal to X, and what is l.i.m.? l.i.m.
Is limit in mean square.
Then this convergence that X n converges to
X that we ah denote by this or this is one
way or we can write in this way. So, ah therefore,
what is mean square convergence? It is the
convergence of the mean square error sequence.
It convergence converges to 0, if it converges
to 0, if it converges to 0 as n tends to infinity
then we say that X n converges to X in the
mean square sense.
We will see one example X n is defined by
this X n are takes n with probability 1 by
n square and X n takes 0 value with probability
1 minus 1 by n square. So, X n is a ah random
uh sequence of random variable ; where each
X n can take 2 values 0 and n with this probabilities
ah. We see that this probability as n tends
to infinity this probability will go down
to 1 and this will go down to 0.
Now, question is whether X n converges to
X is equal to 0. So, this probability become
one as n tends to infinity, but the we want
to examine that ; whether it converges in
the mean square sense . So, what we will do?
E of X n minus X whole square . So, this is
same as E of X n because we X is 0, this is
a this type of ah random variable, it is a
deterministic quantity, it is known as the
degenerate random variable. So, E of X n minus
X whole square is same as E of X n square.
That is equal to E of X n square .
Now, what is E of X n square, that is equal
to X n square can take 0 square with this
probability 0 into 1 minus 1 by n square.
Plus, it can take X n square so, it can take
n square with probability 1 by n square . So,
these 2 get cancel so, ah this is equal to
1. So, limit as n tends to infinity of E of
X n minus X whole square is equal to 1, which
is not equal to 0. Therefore, ah X n does
not converges to X is equal to 0 it does not
convert to X is equal to 0, in the mean square
sense. So, it does not converge, because E
of X n minus X whole square as n tends to
infinity is not equal to 0.
We will consider another example X n is equal
to 1 with probability 1 by n, n equal to 0
with probability 1 minus 1 by n . We see that
0 probability of 0 is increasing as n tends
to infinity it will approach one . So, question
is whether X n converges to X is equal to
0 in mean square sense. So, how we can prove
this? A of X n minus X whole square , that
is equal to E of X n square because X is equal
to 0 E is equal to 1 into 1 by n plus 0 into
1 minus 1 by n. So, this is equal to 1 by
n.
So, what happen limit of a of X n minus X
whole square as n tends to infinity is equal
to limit of this as n tends to infinity is
equal to 0. Therefore, X n sequence in this
case converges in mean square sense to X is
equal to 0 ah.
Now, suppose X n is a sequence of random variable
and X is a random only one random variable
. So, that we have ah considered that X n
this sequence converges to X in the mean square
sense.
Then we want to show that E of X n that mean
it is a deterministic quantity now it is a
deterministic sequence sequence of real number
it converges to E of X . Similarly, E of X
n square this is the mean square value, it
will converge to E of X square. How we can
prove this? Consider the absolute the difference
between E of X n minus E of X.
Now, I can take expectation outside. So, that
that way it will be of X n minus X. Now ah
this I can consider ah this square root of
square of this quantity. So, square root of
E square of X n minus X. Now I know that ah
suppose E of X square is ah greater than equal
to E of X whole square, why? Because I know
that you have X square minus ex whole square
it is the variance . So, that way ah now this
quantity is less than equal to square root
of E of X n minus X whole square.
Now, what happen as n tends to infinity? Because
I know that this sequences m.s. convergence.
So, this term as n tends to infinity it will
go down to 0. Therefore, if I have to take
the limit of this E of X n minus ex as n tends
to infinity that will become 0. Because this
quantity go down goes down to 0 . So, that
way if X n converges to X in mean square sense,
then it is mean sequence will also converge
to mean of X. The same is true for a of X
n square.
Now, let us consider E of X n square minus
E of X square. That is E of X n square minus
X square to E outside . Now if I factorize
this, that is equal to ah mode of this mode
is there, E of X n minus X into X n plus X.
Now we have a famous inequality what is known
as the Cauchy Schwarz inequality what does
it say suppose I have a ah vector X and vector
Y, it is mean it ah if I consider the null
product and then take the magnitude. Then
ah it is less than equal to norm of X norm
of X into norm of Y .
So, in this case, that inner product operation
is defined through this joint expectation
operation. So, here we have a square here.
So, inner product of XY square is less than
equal to norm of X into norm of Y. If I take
this square root then inner product of XY
ah inner product of XY, that is less than
equal to mode of inner product of XY is less
than equal to square root of norm of X into
norm of Y. So, that we apply here so, ah what
we get. So now, it is a norm of E of X and
minus X into X n plus X. So, that is the joint
expectation of 2 quantities . Therefore, ah
we can apply the Cauchy Schwarz inequality.
So, it will be square root of norm of this
quantity E of X n minus X whole square. Similarly,
norm of this quantity is E of X n plus X whole
square .
Now, what happen as n tends to infinity, limit
n tends to infinity? Now this quantity will
go down to 0. Therefore, whatever this value
ultimately this expression will become 0.
So, that we have ah what we have learned?
Therefore, that E of X n square this sequence
also converges to E of X square. So, if X
n sequence converges to X in the mean square
sense, then the corresponding mean sequence
and the mean square sequence they will also
converge. So, ah these 2 important properties
are very useful property for application point
point of view.
Now, we discussed about the convergence of
mean square convergence of this sequence . Now
in the case of deterministic sequence, the
sequence of real number we have one important
result that is Cauchy criterion of convergence
. So, the same concept can be adapted here
if X n in the sequence of random random variables,
then X n converges in m.s. if and only if
E of X of n plus m minus X n whole square
tends to 0 as n n tends to infinity for all
m greater than 0. So, this is the concept
.
Now, here without knowing the limit what is
the mean square limit of X n, without knowing
that also we can ah we can prove whether this
sequence X n converges to some unknown random
variable X or not . So, the result this ah
we have consider the ah mean square value
of different between X of n plus m and X n.
So, this mean square value this mean square
value should go down to 0 as n tends to infinity.
Ok so, that means, if we consider for any
m we consider the mean square error suppose
X n X of n plus m minus X n whole square .
Now, this quantity will go down to 0 as n
tends to infinity, then only we can say that
X n converges in m.s. to some unknown random
quantity X. For example, you can consider
this problem again , probability of X n is
equal to 1, that is suppose is equal to 1
by n, probability of X n is equal to 0 is
equal to 1 minus 1 by n. And we recently assume
that it is a sequence of independent random
variable.
So, we do not know ah suppose that it converges
to X equal to 0. Without that also we can
prove this how we will prove E of X n plus
m X of n plus m, minus X n whole square. This
quantity is equal to E of X of n plus m whole
square plus E of X n square minus 2 E of X
n plus m into X of n. So, this quantity we
can write ah and this ah E of X of n plus
m whole square, X of n plus m square plus
E of X n square minus now this one I can write
as E of X n plus m into E of X n ok.
So, this quantity now what is this quantity?
X of n plus m, suppose it takes value it will
take one with this probability. So, 1 into
1 by n plus m, because X of n plus m whole
square. So, this will be one divided by n
plus m. Because other quantity is 0 similarly,
this will be equal to 1 by n . This quantity
will be E of X of n plus m that is ah one
into 1 by n. So, 2 into 1 by n plus m and
similarly this quantity ah the E of X n it
will be equal to 1 by n.
So, this is the expression and what happens
ah? Limit of this quantity limit E of X n
plus E of X n plus m minus X n whole square,
as n tends to infinity because n is in denominator
for all the expression therefore, this quantity
will go down to 0. Therefore, we can say that
ah this sequence this sequences is a convergent
sequence in the mean square sense. So, that
way we can apply the Cauchy criterion to see
whether a sequence is mean square convergent
or not.
Now, mean square ah convergence is a very
important concept, because we are ah E of
X n minus X whole square suppose X is the
actual quantity X n is some observed quantity
suppose, then E of X n minus X whole square
will represent the mean square error. So,
that way in iterative algorithm suppose whether
that mean square error converge or not that
we can ah prove using this concept of mean
square convergence.
Next we will be considering about convergence
in probability. So, how do I define this?
Suppose X n is a sequence of random variable.
Now we can define a sequence of probability.
Suppose, there is an epsilon , epsilon greater
than 0 it is a arbitrarily small number. Suppose
we are considering this probability, probability
of X n minus X it is deviation. So, absolute
value of this deviation is greater than epsilon.
What is the probability? Now this probability
for a given epsilon is a function of n. So,
it is a function of n therefore, it is a real
sequence. Now, if this sequence ah is convergent;
that means, probability of X n minus X absolute
value of that is greater than epsilon, if
that probability sequence this probability
sequence if it goes down to 0 as n tends to
infinity , then we say that X n converges
to X in the sense of probability . So, this
sequence X n is said to converge to X this
is a random variable in probability, if this
sequence of probability this sequence of probability
is convergent that is probability of X n minus
X absolute value of that is greater than epsilon
.
So, the probability that absolute value of
X n minus X is greater than epsilon goes down
to 0 as n tends to infinity. We write this
is the notation sequence X n converges in
probability to X. And so, that way we denote
the convergence in probability. So, X n converges
to X in probability that is denoted by this
notation. So, this is a weaker sense of convergence,
but how it is ah related to other ah concepts
like almost sure and mean square convergence
that we will see .
Now, we will see an example first suppose
similar example X n takes 0 with probability
1 minus n 1 by n square. And X n takes value
n with probability 1 by n square. Now ah examine
if X n converges to X is equal to 0 in probability.
So, we have to consider solution suppose for
any epsilon greater than 0 what we can write
probability of absolute value of X n minus
X that is greater than epsilon. That will
be probability of X is equal to 0, probability
of X n mod of X n is greater than epsilon
.
So now whatever epsilon we consider that mod
of X n greater than epsilon, that will be
ah the probability that X n is equal to n.
Because other value of X n is equal to 0,
that is equal to probability of X n is equal
to n . So, what is this probability ? So,
this is equal to 1 by n square. So, therefore,
limit of this probability sequence probability
of X n minus X that is greater than epsilon.
So, that probability is equal to as n tends
to infinity that will become 0. Therefore,
we say that ah X n sequence that X n sequence
converges to X is equal to 0, this is a random
variable in the sense of probability.
Ah Now ah we will examine the relation between
mean square convergence and convergence in
probability. This is the theorem if X n converges
to X then X n converges to X n probability
. X n converges to X in the mean square sense
implies that X n converges to X in probability.
So, this is the theorem suppose if we can.
So, that X n is convergent in mean square
sense , then X n will be convergent in probability
sense also.
So, proof is simple what we what do I proof
ah that is ah ah given that limit of X n minus
X whole square, if we take the expected value
of X n minus X whole square and take limit
as n tends to infinity. That will become 0,
because X n converges to X in the mean square
sense . So, this is given .
Now, for any epsilon greater than 0 , what
we will get? Probability of mod of X n greater
than epsilon. So, that will be equal to because
ah we can consider it in terms of the square
quantity. So, that way probability of X n
minus X whole square is greater than epsilon
square, because just we are squaring ah both
quantity. So, this probability and this probability
is same.
Ah Now this is a ah positive quantity E of
X n minus X whole square. So, this is a positive
quantity. So, this positive probability that
this ah positive quantity Y is suppose greater
than some ah quantity a. So, probability of
Y Y is a positive quantity it is positive
Y is positive Y greater than a. So, that will
be less than equal to E of Y Y a. So, this
is the Markov inequality. So, if I apply here
it will be E of less than equal to E of X
n minus X ah whole square divided by epsilon
square. So, this is the ah relationship.
Now, what we know that this quantity as n
tends to infinity this goes down to 0 . Therefore,
if I ah consider limit as n tends to infinity
of this probability sequence , then it will
become limit of E of X n minus X square that
is equal to 0. So, less than equal to 0 probability
cannot be a negative. So, this is equal to
0. Therefore, what I what is my conclusion
the sequence of X n converges to X in probability
also. Mean square convergence imply convergence
in probability, very important result.
The converse of the theorem is not generally
true X n convergence in probability to X does
not necessarily imply that X n converges in
mean square sense to X. Suppose X n is a sequence
of random variables with p of probability
of X n is equal to 0, probability of X n is
equal to 0 is equal to 1 minus 1 by n square.
And probability of X n is equal to n is equal
to 1 by n square.
So, in this case ah, we see that this probability
sequence probability that X n is equal to
0 this is converging into 1. So, ah if we
see if 0 is the X equal to 0 if the limit
in probability. So, that way probability that
X n minus X mod of that that is greater than
epsilon. So, in this case ah X, X is equal
to 0. So, probability of X n mod of X n greater
than epsilon. So, the this is equal to probability
of X n equal to n , that is equal to 1 by
n square. Therefore, limit probability of
X n ah minus X that is greater than epsilon
. So, this probability will go down to 0;
imply that ah, this is sequence convergence
in probability converges in probability. So,
by that X n converges in probability to X
is equal to 0.
Now, ah considering the mean square convergence
we see that E of X n minus X whole square,
that is equal to E of X n minus X is equal
to 0. So, that way it will be E of X n square
since X is equal to 0 . So, that is equal
to 0 into 1 minus 1 by n square plus n square
into 1 by n square so, this is equal to 1.
Therefore, limit E of X n minus X whole square
as n tends to infinity. Therefore, limit therefore,
limit limit of E a E of X n minus X whole
square as n tends to infinity, that is equal
to E of X n square limit n tends to infinity
that is equal to 1 which is not equal to 0.
Therefore X n does not converge in the mean
square sense to X is equal to 0. So, ah we
see that in this case ah X n converges in
probability to X is equal to 0, but X n dos
not converges in mean square sense to X is
equal to 0. So, ah what we have establish?
Therefore, convergence in mean square sense
implies convergence in probability.
Similarly, we have suppose if X n is convergent
to X in the almost sure sense convergence
X n converges to X almost sure, then X n converges
to X in probability also. So, almost sure
convergence imply ah imply convergence in
probability. S convergence implies convergence
in probability . So, this also we can prove
ah. Now here in place of epsilon, we can we
consider 1 by m; where m is a ah arbitrary
large positive integer .
Now, ah we know that X n is almost sure convergence
to X. What does it means? Probability of this
l.i.m. sup of this deviation. So, X j's minus
X s greater than 1 by m as such that absolute
value of this deviation is greater than m.
So, that you will take the union from j is
equal to m to infinity. And then ah this sequence
we will take the intersection of all from
n is equal to 1 to infinity . So, this is
the ah actually this is the l.i.m. sup l.i.m.
sup of this deviation sequence .
Now, I know that ah that inside part ah that
union part this is the decreasing sequence.
So, ah we can apply the continuity theorem.
So, that way ah what we will get is that,
this limit of this, suppose this will be same
as ah what you will show here is that ah probability
of this l.i.m. sup is same as probability
of the inner event as n tends to infinity.
So, this concept we will use ah now let us
see that probability of this quantity is greater
than 1 by m.
Ah Now ah this is single event therefore;
this probability will be less than if we consider
ah some union of event . Because this is one
event there. So, that way that probability
will be ah bigger this right hand side probability
is bigger therefore, this probability is less
than this quantity.
Now, I know that this quantity is same as
ah this quantity, because ah this if I consider
this sequence as I told earlier this is a
decreasing sequence. Therefore, probability
of this will be same as this probability as
n tends to infinity. So, therefore, this quantity
I can substitute by ah a quantity in terms
of l.i.m. sup, ok . But given that X n converges
almost sure to X therefore, this quantity
is equal to 0. Therefore, this quantity that
probability there this absolute deviation
from the random variable X ah is greater than
1 by m is ah going down to 0 as n tends to
infinity. Therefore, this sequence X n sequence
converges to X in probability. So, almost
sure convergence implies convergence in probability.
We will consider one example X n be a sequence
of random variables with probability of X
n is equal to 1 is 1 by n and probability
of X n is equal to 0 is equal to 1 minus 1
by. So, this sequence ah we can prove that
this sequence suppose this sequence is convergent
in probability in probability . Because probability
that X n is positive because this is 0 is
the random variable. So, that X n is positive,
that is only one case X n can take one and
that probability is 1 by n. And this probability
will go down to 0 as n tends to infinity.
Therefore, this sequence is convergent in
probability .
But whether ah it is convergence almost in
almost sure sense ah. That we can see this
condition suppose ah here we will consider
the sequence of independent random variable.
That also we will be considering. So, in that
case ah we can so that ah that deviation sequence
probability that mod of X n minus X is 0 X
0 , that is greater than 1 by m. So, if I
consider this as ah ah from n is equal to
ah 0 to n is equal to 1 to infinity, if I
consider this . So, this now this is the ah
what is this probability that is equal to
1 by n. So, summation 1 by n going from n
is equal to 1 to infinity .
So, ah now this will diverge to infinity.
Therefore, ah this probability that there
will be this divergence ok, that probability
will be equal to 1; that means, this sequence
X n does not converge ah in almost sure sense
to X is equal to 0. So, it does not converge.
So, what we have seen? That, ah though ah
we have the theorem that convergence almost
sure imply convergence in probability, but
other way it is not true. Convergence in in
probability does not imply convergence in
in almost sure sense, ok. So, this is ah another
important result that convergence in probability
does not generally does not imply convergence
in almost sure sense.
So, we have considered 2 important weak convergence
concept convergence 
in mean square . What does it ah say? It says
that E of X n minus X whole square goes down
to 0 as limit of this as n tends to infinity
that will be equal to 0. Convergence then
X n is convergent in mean square sense .
Then convergence in probability what does
it means ? That it says that, probability
sequence probability that X n minus X mod
of that that is greater than an epsilon. So,
this probability sequence limit of this probability
sequence as n tends to infinity is equal to
0 .
So, ah in the case of convergent in mean square
sense that, mean square error sequence that
goes down to 0 as n tends to infinity. So,
this is equal to 0 as n tends to infinity.
In the case of convergence in probability,
we are considering the probability of the
absolute deviation that is mod of X n minus
X is greater than epsilon. This also goes
down to 0, for any epsilon as n tends to infinity.
Then we say that X n converges to X in probability
. And there is the implication result what
we have shown that suppose this is convergence
m.s. convergence and then we have convergence
in probability . So, this will imply this
m.s. convergence we will always imply convergence
in probability.
And earlier we discussed about almost sure
convergence, that is the stronger concept.
So, this almost sure convergence here suppose
m.s. convergence and convergence in probability
. So, this will also imply that convergence
in probability m.s. convergence will always
imply convergence in probability m.s. convergence
will also imply convergence in probability
. So, that way ah and this concept will imply
this this concept will imply this, but other
way it is not true.
So, other way it is not true means generally
convergence in probability will not imply
m.s. Convergence or convergence in almost
sure sense . So, we discuss about toward the
weak convergence concept. And ah the inter
relation between as convergence m.s. convergence
and convergence in probability.
Ah We have to consider another weak concept
that is convergence in convergence in distribution
. So, in this case ah, this ah distribution
function sequence that will be again a real
sequence that sequence will be converging
to some distribution function of some random
variable X. So, that way we will ah discuss
about ah this weak concept also weak convergence
concept convergence in distribution. And then
we will see some of the application of all
these convergence concept ah as convergence
m.s. convergence convergence in probability
and convergence in distribution . That we
will be discussing ah in the next class and
what is the implication ah between these 2
that also we will be discussing .
Thank you .
