Let us continue our discussion on Convergence
of Sequence of Random Variables. We have talked
about our different notions of convergence.
In the last class, we looked at Cauchy criteria.
What is the other thing we looked in the last
class, other than Cauchy criteria? So, we
looked at some other means of other conditions
or other properties that one can verify, to
check whether a sequence converges to a sequence
to us to a limiting random variable.
One was this correlation property of the random
variables. So, we move on now, like we want
to now study, fine. We have a sequence of
random variable whether they converge to some
limiting random variable. So, often instead
of looking at the random variable itself,
we may be interested in looking whether the
means of this random variable converges to
some value. So, when I look at the means of
each of these random variables, I will end
up with a deterministic sequence.
After taking expectation of each of this random
variable, I will have a deterministic sequence
and it converges. So, and if it converges,
now, I want to understand when the mean will
converge? Or are there any properties that
will help me to infer from the properties
of convergence of the random variables in
other, what you have already studied like
convergence and probability.
So, let us say that I have Xn converging to
X in probability. So, what does this mean?
This means basically limit as, sorry. So,
this basically we said that probability that
is equals to 0. Now, suppose if this happens,
can I say that expectation of Xn goes to expectation
of X? What I am basically asking is? So, what
is X here? Expectation of X is nothing but
expectation of limit as Xn and this limit
is what is in the probability sense, whether
this happens or what I am been saying to ask
whether this is same as limit as n tends to
infinity of expectation of Xn.
When I said this, this is what exactly I meant.
Whether expectation of Xn converges to expectation
of X is basically saying whether expectation
of Xn, as n goes to infinity is equal to expectation
of X. Yes, we have said but we are doing this
we are asking, we are asking this question?
Whether this holds? We are not right now saying
that this is true, we want to see whether
this holds, if at all it holds when does this
hold?
So, when you studied some examples, when you
are studying convergence of random variable
in, in probability sense and the means squared
sense. We came across an example where it
said that it converges in probability, but
it did not convergence means squared sense.
We, we had an example like that, because convergence
in probability does not always imply convergence
in mean squared sense.
That happened in that example, because it
so happened that, even though the example
I had their Xn's they are putting values on
a smaller and smaller intervals as I put n
goes to infinity, but their amplitude was
also exploding. So, you remember like, we
had this example where, so, it was like this
and this was some sequence what was we called
it an as n increases, this was putting on
a smaller interval but it was amplitude was
also increasing, and we had argued that unless
an grows much smaller than n, this guy will
diverge, I mean, the mean squared error will
diverge and this will not converge in the
mean squared sense.
So, similar behavior happens here, like that
is what we want to capture. If we are looking
at the expectation of the random variable,
even though it may be putting that mass on
a smaller and smaller interval, but, it may
take a very large value, right, even though
because of that, it may converge in probability
sense in this fashion that limit and this
may be almost same everywhere except for a
small interval differing by epsilon or like
they may be, they may be different at only
some small interval, but on that interval,
my Xn may be taking a very large value because
of that expectation could be very large there
and in that way, we cannot, we cannot say
that always, this kind of behavior happens.
To make this more concrete let us look at
an example. So, let us say U is a random variable,
okay. Now, I am going to come. So, let us
say I have random variable U, I have a sequence
of Ai's, which are events, and these events
are such that their probability goes to 0
as n goes to infinity. Okay. And I am also
going to look at this sequence bi. These are
another, set of sequence of numbers, some
numbers that are given to me.
Let me see I, so and I am going to assume
that they are non-zero. So, this is I am starting
with and I am going to define a sequence of
random variables based on this in the following
fashion. So, this U does not depend on any
n it is just like one random variable U. I
am going to define for each n, a new random
variable, which is the sum of these two random
variable U and, this bn is thus, deterministic
quantity which is coming from this given sequence
and it is multiplied by this Indicator function
which depends on An and An is the event that
I am, I am, I am assuming it will be such
that it probably goes to 0.
Let us say I have such a setup. Do you understand
what I mean by indicator of An, fine. Now,
if I am going to look at his expectation what
his expectation is? I will assume that with
finite means. Okay. Plus, what is this? What
will be if I take expectation? This will be
bn into; it is going to bn expectation of
indicator function. What is then? It will
be probability of An. So, it looks like its
expectation for any n is going to look like
this.
So, now let me ask this question. I have a
sequence of random variables like this, does
this converge in probability? So, to verify
that what I need to do, probability that what
you guess it should be converging to if at
all it converges to U. So, let us say our
guess is. Let us assume I am going to take
to be U. 
What is this quantity? If I can show that
that as n go to infinity, this probability
goes to 0, then I know that Xn converges to
U in probability sense.
Now, should that or see that this probability
is same as Xn is not equals to U. See, when
I say Xn is not equal to U that means the
difference is going to be some non 0 quantity.
So, it could be either greater than epsilon
or it could be much greater than that, that
is what this is going to be an upper bound,
fine? So far? Now, come back to this. Now
what is the probability that Xn is not equal
to U?
It is going to be, so, probability that Xn
are not going to be equals to U, that is enough
that if this quantity happens to be positive
then this going to be different and when this
count is going to be positive when this is
true, and what is that probability? That probability
is nothing but the probability of event An.
So, the way I have constructed this, Xn not
is equals to U is nothing but probability
that, of An.
So, the, the problem is that they are not
differing is that this quantity is going to
be positive, this quantity is positive is
nothing but this An has to be this, this indicator
function has, this condition or this indicator
function has to satisfy and that is going
to be satisfied probability An and what is
our assumption? We have said that I will come
up my assumption said that this An is such
that this guy goes to 0.
So, then we, what? So, what we have shown
that; so what we just have shown is that Xn
converges to U in probability sense. Now,
let us come back to this. So, if Xn converges
here to U in probability, fine. Now, I wanted
to ask the question, can I also do this and
can I also say that expectation of Xn converges
to expectation of X, X in this case is going
to be U. Because that is the limiting distribution.
But here if you focus on this even though
this probability of An's are going to 0 as
n goes to infinity, I could choose bn's such
that this product does not go to 0. So, in
that case, this expectation of Xn need not
be the same as expectation of U as n goes
to infinity. They, they can be different.
So, as you say that even though Xn can converge
to some distribution in probability, but that
does not automatically allow us to take expectation
of the sequence to be the expectation, the
limit of expectation of the sequence to be
the expectation of the limiting random variable.
So, then the question now is: When does this
happen? We want to now, look at the conditions
when does this happen? Yeah, yeah.
So, indicator of an event, expectation of
indicator of event is what? Is the probability
that event itself? How can you write this?
Just write it into 0 and just expand it, so
expectation of this. Only the one term remains
the other term is 0, the one time that remains
is exactly this 1 into Pr of An. This one?
Let us look at this event Xn not equals to
U. That means the difference between them
is not 0. It is other than 0, it could be
positive or negative, then you are going to
look at the absolute value that is going to
be the difference is going to be greater than
0. What you are saying, all you are asking
is, this difference is going to be greater
than 0. That could be either greater than
0, greater than epsilon or it could be much
more than that. Exactly. So, this event is
going to be subset of this event that is why
I am going to get an upper bound, fine.
So, then I am going to now state couple of
results, which actually helps to do this kind
of interchange. What I mean by interchange
is exactly this. The first one is called bounded
convergence theorem. See, some of these results
are you may not be able to directly relate
it to where they come in practically, but
some of these results are useful to state
some results, some, some of that you are going
to see later in the class today, that gives
us very practical, some of uses some intuition
about some of the practical things.
So, these are like some intermediary building
steps that we need to understand, to understand
some larger result, for example, later in
the class today I am going to talk about Central
Limit Theorem, that is one of the important
result and which has many practical implications.
So, to understand that we have to understand
these results. So, at every result, do not
try to connect it with practical things, then
you may be lost but just try to follow what
we are trying to define and then eventually
where we are kind of going to connect it.
Let, this be a sequence 
such that there exist some L. So, this, this
theorem tells gives us the first condition;
it says that, suppose you have a sequence
and such that you have some L which is finite
and all my random variables, their absolute
values is bounded by this random variable
with probability 1, then convergence in probability
to a random variable X implies that their
expectations, expectation of the sequence
of random variable converges to the expectation
of the limiting random variable.
So, this is kind of this is a very intuitive
statement. So, that is why it is called bounded
convergence. What you are saying is your random
variables are always bounded. That naturally
means that even when you are limiting random
variable cannot be unbounded. So, if you have
everything bounded then there is nowhere that
in the sequence you will incur a case where
something explodes.
So, because of that there is nothing absurd
behavior when I have this boundedness and
everything goes well. Okay. Let us look to
proof of this. So, then I make this kind of
assumption that all the random variables are
bounded like this, where I made such as such
case again. Did I use such an assumption earlier
also? Where?
If Xn converges to Xn in mean squared sense.
In means squared sense, when I had a mean
squared set I wanted all the second moments
of all the random variables to be bounded
there. So, if I have a random variable like
this and if L is finite and if I take the
second moment, will it be still bounded and
that limit is going to be what? Instead of
L it is going to be L square. So, this condition
and I had, when I said convergence in probability
implies convergence in distribution, but convergence
in probability does not always imply convergence
in mean squared sense.
But I said that convergence a probability
implies convergence means squared sense under
some condition, what was that? Yeah, so, if
my sequence are bounded by some random variable,
so here is it; can I say that, here convergence
in p already implies convergence in mean squared
sense? Yes, right because that Y is nothing
but L in this case and we have already said
that…
Just L square, L square should be finite,
fine. So, and we have already said that so
under this condition, we know that if Xn converges
to X in probability it already converges to
mean squared sense and in the mean squared
sense, when I discussed it, I already said
that the limiting random variable, the second
moment is already finite.
That was a consequence which I asked you to
verify, I do not know you guys did. So that
came from Triangular inequality. So, by the
same logic, we can also say that here, this
X is going to be bounded, its expectation
is going to be bounded. So, this is also going
to be bounded. Now, let us understand.
So, you can verify that I am just leaving
you this, that X is going to be also going
to be L, probability 1; whatever that limit
is. After this, let us focus on this inequality.
Take any epsilon greater than 0. Now, my claim
is this X minus Xn, I can write upper bounded
in this fashion. So, this is the main inequality
I need to make this final claim.
So, let us see, we understand this inequality.
So, taken epsilon greater than 0, if it is
such that this X minus Xn, the absolute value
is greater than epsilon, greater than equals
to epsilon, if this is true or like let us
say this is not true that is this difference
is upper bounded by epsilon, this term is
already 0, right. And the then this condition
already true like X minus Xn is upper bounded
by epsilon that is what I have already assumed.
In the other case, where let us say this holds.
Well X minus Xn is greater than epsilon, it
is going to be at least anyway epsilon. It
is going to be plus something else and what
is that something else? We can always write
X minus Xn as upper bounded by X plus X of
n. And both of them are upper bounded by L
with probability 1.
So, I could read them as upper bounded by
2L the probability 1. So, just verify that
is why I could write this will be greater
than 2L at most. I mean, see like this X minus
Xn, I could have already written. This is
upper bounded by 2L, but I am expressing in
terms of this epsilon because this gives me
a tighter bound. That is if this condition
holds, yes, 2L is there 2L is there; if this
condition does not violate, then epsilon is
the bound that is why tighter bound.
Anyway, so, I have this inequality here. So,
now let us see what I want? I want to, now
I want to find expectation. So, what is the
meaning of expectation of Xn goes to expectation
of X that means, if I am going to look at
the absolute difference of this expectations
this should go to 0 as n tends to infinity
or alternatively what I can show as if this
is upper bounded by epsilon for some n, which
is sufficiently large, right, for all n after
some point. So, now fine this is there this
I could write it as expectation of X minus
Xn, why I could do this?
Linearity.
Because of the linearity of the expectation.
And now, I could do that. Now, I have taken
this absolute term inside the expectation,
is this true? Because the differences I have
made absolute so my expected value is going
to be larger, fine. Now, I am going to bring
in this bound which I applied here. And what
is this is going to be? So, if I use this
bound, this is going to be epsilon plus 2L
and expectation of this indicator.
So, that is going to be. Right. Now, I know
that this guy goes to 0, this probability
goes to 0. And I know that because of that,
then I can say that there exists some and
n0 such that for all n greater than or equals
to n0 I can replace this guy by maybe there
exists for all n greater than n I can write
it as this there is going to be less than
or equals to, so I can always write it as.
So, you give me epsilon, I will take epsilon
by 2L and I will look after which point this
guy is going to fall below epsilon by 2n.
I can always, there always exists such an
n. So, now you should do that for sufficiently
large n, this guy is going to be upper bound
plus 2 epsilon. This I know this guy goes
to 0, right? As n goes to infinity, ok. Whatever
epsilon it is or whatever on you again it
goes to 0.
Now, what I will do is whatever epsilon you
give me I will be looking at epsilon by 2L
and I know that after some n0, this guy is
going to be below epsilon by 2L. It is because
this guy is going to 0. So, I just plugged
this quantity here for all n which are greater
than this n, then it becomes 2 epsilon, it
is just like… What?
We can take anything.
Yeah, you can take anything I just want to
bring it simplifying all this to epsilon.
Now it is clear right like now I am more familiar
territory. Now, I can say that, I have shown
that this difference is upper bounded by 2
epsilon for all n greater than n0 and this
and n0 is a function of epsilon. And now this,
if this holds, if I can do this for every
epsilon, then what is that? What does this
imply? This epsilon is arbitrary. So, then
this is exactly the definition that this Xn
converges to X.
So, this implies, so, I am saying that this
implies this, are you convinced or not? So,
what I could show? What can now think of this
as some this was an sequence, and this is
the limit, what I have shown is an minus a
is going to be upper bounded by 2 epsilon
for n sufficiently large and this this I am
doing for any epsilon that you are given to
me. So, that means that is what the definition
of the limit, an converging to a and I could
do that, because I am exploiting the fact
that I already know that Xn converges to X
in probability.
I am just using that convergence to claim
that this guy converges. So fine. So, this
is one simple thing, like if I know that my
bound random variables are bounded, and if
that may and my random variable convincing
probability, then I can do this interchange
of the expectations. Yeah, so what is the
other things?
Not necessarily, it is not only if and only
if condition. If this holds then we say we
can do this but if we can do this at all,
I do not know that is it is that always implies
that such a thing implies.
So, let us say this other things which tries
to generalize this bit. See when I try to
state this, I wanted this Xn’s to be bounded,
right and this bound was a deterministic bound.
But you can relax this and say that instead
of the deterministic bound, I can have this
bound with some prob, using a prob, using
a random variable. So L I can replace by random
variable Y, if such a Y exists which always
dominates Xn, then also it is possible.
So, let me make that. So this is called, so
this is just a slight generalization of this
bounded convergence instead of boundedness
now you look domination. We are going to say
that these sequences such that they are all
dominated with probability 1 by random variable
Y and is dominating random variables such
that its mean is bounded and then if I have
the 
sequence converging to some X infinity in
probability then, I can do this interchange.
So, it is same as this except for the fact
that I am replacing this L by this Y which
dominates the sequence, but that Y are further
one that it has a finite expectation. So,
if I can find such a random variable which
has a finite mean dominates my sequence and
then I can interchange this limit if my sequence
converges in probability. So, we are not going
to prove this, but just take this like you
should understand how to apply this results.
So, before you apply, you should better check
all these conditions.
So, the last one called monotone convergence
theorem. So, this is as the name indicates,
this is analog another version of monotone
convergence we have already come across deterministic
sequence. So, what do you know about a deterministic
sequence when it is monotonically increasing?
So, let us say I have a sequence, deterministic
sequence, let us talk about deterministic
case.
If it is always increasing like a of n plus
1 is going to be greater than equals to a
of n, what happens to limit an? The limit
exists to convergence, what? Depends on n,
why? So, any sequence which is monotonically
increasing necessarily convergence, right?
Either finite one or it could be like even
the limit could be unbounded. It at least
it does not diverge, right?
So, if I have a monotone let us say I have
let us say these are a1, a2, 3, 4, like that,
right, my and this is my sequence, if my sequence
is like this increasing it is increasing,
increasing that either it blows up and goes
to infinity or like it saturates at some point,
right? So, it always converges. So, similar
analog.
If you do not know, how will you unbound it?
So, this is the convergence in the extended
domain. So, we allow the limit to be infinity
as well. So, it is not necessary that when
we talk about convergence, the limit has to
be always finite even the limit could be infinity,
right? It is just like that it is just that
it does not happen that while it is going,
going like at some point I come down. So,
this violates the definition of my convergence,
right?
So, if it is always increasing, increasing
at some point either it goes to infinity in
that case it is infinite is the limit or it
saturates at some point then the finite in
that case that saturation point is the my
limit. Let me take this, I have this n; n
greater than or equals to 1 this is a monotonically
increasing sequence, right? Where does this
converge? Converge like this, this is the
extended notion of convergence.
No, this was deterministic case also. I am
giving you a deterministic case, yeah, right?
I have this n, why I like the sequence an,
this an is simply n, this is convergence but
it converges to infinity, fine. So, I am not
defining the sequence in this form. So, I
have my random variable sequence such that
if you are going to fix a sample point on
that sample point this forms a monotonically
increasing sequence, and then defined my limiting
random variable for each omega to be the limit
of this.
So, now this Xn of omega is deterministic
sequence, right? And since this Xn of omega
monotonically increasing, this, indeed exists;
maybe it could be, limit could be infinity.
What is the term for that? With possible value
of infinity. So, if this happens then I can
always say my expectation I can interchange
like this and here it really do not need to
first check. I mean, I do not really need
a convergence in probability, if this monotonicity
property holds that is it, like if I can check
this, then I can always.
Infinity of omega become infinity.
Yeah.
What will be expectation of Xn, it will also
be inifity.
Yes, but provided that sample point has non
zero probability. If that omega has 0 probability,
even if it is infinity, mean, that becomes
a bit weird to define, right? Like this has
infinity, but its probability could be 0.
But in that cases, we have to define its value
to be the product of 0 and infinity to be
0, and then we can continue with that, ok?
So, fine. So, these are three theorems called
bounded convergence, dominated convergence
and monotone convergence. You should know
and understand when you can use them.
