We will now go through two
examples of convergence in
probability.
Our first example is
quite trivial.
We're dealing with a sequence
of random variables Yn that
are discrete.
Most of the probability
is concentrated at 0.
But there is also a small
probability of a large value.
Because the bulk of the
probability mass is
concentrated at 0, it is a good
guess that this sequence
converges to 0.
Do we have, indeed,
convergence in
probability to 0?
We need to check
the definition.
So we fix some epsilon, which
is a positive number.
And we look at the probability
of the event that our random
variable is epsilon or more away
than what we think is the
limit of that sequence.
We look at that probability.
And in this example, it is equal
to 1 over n, which goes
to 0 as n goes to infinity.
And this verifies that, indeed,
in this example, Yn
converges to 0, as n goes to
infinity in probability.
Now, we make the following
observation.
If we are to calculate the
expected value of this random
variable, what we get
is the following.
We get a value of 0 with this
probability, no contribution
to the expectation.
But we also get a value
of n squared with
probability 1 over n.
And so the expected value is
equal to n, which, actually,
goes to infinity, as
n goes to infinity.
So we have a situation where
the sequence of the random
variables converges to 0.
But the expectation does
not converge to 0.
In fact, it goes to infinity.
And this example serves
to make the point that
convergence in probability does
not imply convergence of
expectations.
The reason is that convergence
in probability has to do with
the bulk of the distribution.
It only cares that the tail of
the distribution has small
probability.
On the other hand, the
expectation is highly
sensitive to the tail
of the distribution.
It might be that the tail only
has a small probability.
But if that probability is
assigned to a very large
value, then the expectation will
be strongly affected and
can be quite different
from the limit
of the random variable.
Our second example is going to
be less trivial and more
interesting.
Consider random variables
that are independent and
identically distributed and
whose common distribution is
uniform on the unit interval,
so that the
PDF takes this form.
Are these random variables
convergent to something?
The answer is no.
And the reason is that as i
increases, the distribution
does not change.
And it does not to
get concentrated
around a certain number.
The distribution remains spread
out over the entire
unit interval.
But let us look now at some
related random variables.
Let us define Yn to be the
minimum of the first n of the
X's that we get.
So if n is equal to 4, and we
obtain these four values, Yn
would be equal to this value.
Notice that if we draw more
values, then the new values
might be above the minimum, in
which case the minimum does
not change.
But we might also get a value
that's below the minimum, in
which case the minimum
moves down.
The only thing that can happen
is that the minimum goes down.
It cannot go up.
And this gives us
this inequality.
So the random variables
Yn tends to go down.
How far down?
If n is very large, we expect
that we will obtain some X's
whose value happens to be very
close to 0, which means that
Yn will go down to values that
are very close to 0.
And this leads us to conjecture
that, perhaps, Yn
does converge to 0.
This is always the first step
when dealing with convergence
in probability.
The first step is to make an
educated guess about what the
limit might be.
And then we want to verify
that this is, indeed, the
correct limit.
To verify that, what we do is we
fix some positive epsilon.
And we look for the probability
that the distance
of the random variable Yn from
the conjectured limit has a
magnitude that's larger than
or equal to epsilon.
And what we need to show is that
this quantity converges
to 0 as n goes to infinity,
no matter what epsilon is.
Now, because Yn is a
non-negative random variable,
this is the same as the
probability that Yn is larger
than or equal to epsilon.
Now, let us distinguish
between two cases.
If epsilon is bigger than
1, we're asking for the
probability that Yn is larger
than or equal to a certain
number epsilon that's
out there.
But this probability is 0.
There's no way that the minimum
of these uniforms will
take a value that's larger
than some epsilon that's
larger than 1.
So in that case, this quantity
is equal to zero.
And we are done.
But we need to check that this
quantity becomes small no
matter what epsilon is.
So now, let us consider taking
a small epsilon that is some
number that's less than
or equal to 1.
For that case, let us continue
with the calculation.
The minimum is going to be at
least epsilon, if, and only
if, all of the random variables
are at least epsilon.
So this is an equivalent way
of writing this particular
event here.
Now, because of independence,
this is the product of the
probabilities that each one of
the random variables is larger
than or equal to epsilon.
The probability that X1 is
larger than or equal to
epsilon can be found
as follows.
If we have here epsilon, the
probability of being larger
than or equal to epsilon
is the probability
of this event here.
So it's the area of
this rectangle.
The base of that rectangle
is 1 minus epsilon.
And so we obtain 1 minus epsilon
for this first term.
But because the Xi's are
identically distributed, all
the other terms that we multiply
are also the same.
And so the answer is this
expression here.
Now, epsilon is a
positive number.
So 1 minus epsilon is strictly
less than 1.
And when we take higher powers
of a number that's less than
1, we obtain something
that converges to 0
as n goes to infinity.
And that's what we
needed to verify.
Since this is the case for any
epsilon, we conclude that the
random variables Yn converge to
zero in the sense that we
have defined, in probability.
Generalizing from this example,
when we want to show
convergence in probability, the
first step is to make a
guess as to what is
the value that the
sequence converges to.
In this example, that value
was equal to 0.
Once we have made that
conjecture, then we write down
an expression for the
probability of being epsilon
away from the conjectured
limit.
And then we calculate that
probability either exactly, as
in this example.
Or we try to bound it somehow
and still show
that it goes to 0.
