 
In this exercise, we'll be
working with the notion of
convergence in probability, as
well as some other notion of
converge of random variables
that we'll introduce later.
First type of random variable
is xn, where xn has
probability 1 minus 1 minus
over n to be as 0 and
probability of 1 over
n to be a 1.
 
And graphically, we see that
we have a pretty big mess.
1 minus 1 over n at location
0, and a tiny bit somewhere
here, only 1 over n.
So this will be the PMF for x.
 
On the other hand, we
have the sequence of
random variables, yn.
Fairly similar to xn with
a slight tweak.
The similar part says it also
has a very high probability of
being at 0, mass 1
over 1 minus n.
But on the off chance that yn
is not at 0, it has a pretty
big value n.
So it has probability 1 over
n of somewhere out there.
So to contrast the two graphs,
we see at 0, they have the
same amount of mass, 1 over 1
minus n, but for y, it's all
the way out there that has
a small mass 1 over n.
So this will be our Pyn of y.
 
And for the remainder of the
problem, we'll be looking at
the regime where the number n
tends to infinity, and study
what will happen to these two
sequences of random variables.
In part A, we're to compute the
expected value n variance
for both xn and yn.
Let's get started.
The expected value of xn is
given by the probability--
it's at one, which is 1 over n
times 1 plus the probability
being at 0, 1 over
n times value 0.
And that gives us 1 over n.
To calculate the variance of
xn, see that variance is
simply the expected value of xn
minus expected value of xn,
which in this case is 1 over n
from the previous calculation
we have here.
We take the square of this value
and compute the whole
expectation, and this gives us
1 over n, 1 minus 1 over n
plus the remainder probability 1
over 1 minus n of x being at
0, so 0 minus 1 over
n squared.
 
And if we carry out the
calculations here, we'll get n
minus 1 over n squared.
 
Now, let's turn to yn.
The expected value of yn is
equal to probability of being
at 0, 0 plus the probability
of being at n and
times the value n.
And this gives us 1.
The variance of yn.
We do the same thing as before,
we have 1 minus 1 over
n probability of being at 0,
multiplied 0 minus 1 squared,
where 1 is the expected
value of y.
And with probability 1 over n,
out there, equal to n, and
this is multiplied by
n minus 1 squared.
And this gives us n minus 1
Already, we can see that while
the expected value for x was 1
over n, the expected value for
y is sitting right at 1.
It does not decrease
as it increases.
And also, while the variance
for x is n minus 1 over n
squared, the variance for
y is much bigger.
It is actually increasing to
infinity as n goes infinity.
So these intuitions will be
helpful for the remainder of
the problem.
In part B, we're asked to use
Chebyshev's Inequality and see
whether xn or yn converges to
any number in probability.
Let's first recall what the
inequality is about.
It says that if we have random
variable x, in our case, xn,
then the probability of xn minus
the expected value of
xn, in our case, 1 over n,
that this deviation, the
absolute value of this
difference is greater than
epsilon is bounded above by the
variance of xn divided by
the value of epsilon squared.
Well, in our case, we know the
variance is n minus 1 over n
squared, hence this whole term
is this term divided by
epsilon squared.
Now, we know that as n gets
really big, the probability of
xn being at 0 is very big.
It's 1 minus 1 over n.
So a safe bet to guess is that
if xn work to converge
anywhere on the real line,
it might just converge
to the point 0.
And let's see if that is true.
Now, to show that xn converges
to 0 in probability, formally
we need to show that for every
fixed epsilon greater than 0,
the probability that xn minus 0
greater than epsilon has to
be 0, and the limit has
n going to infinity.
And hopefully, the inequalities
above will help
us achieve this goal.
And let's see how
that is done.
I would like to have an
estimate, in fact, an upper
bound of the probability xn
absolute value greater or
equal to epsilon.
And now, we're going to do
some massaging to this
equation so that it looks like
what we know before, which is
right here.
Now, we see that this equation
is in fact, less than
probability xn minus 1 over n
greater or equal to epsilon
plus 1 over n.
Now, I will justify this
inequality in one second.
But suppose that you believe me
for this inequality, we can
simply plug-in the value right
here, namely substituting
epsilon plus 1 over n, in the
place of epsilon right here
and use the Chebyshev Inequality
we did earlier to
arrive at the following
inequality, which is n minus 1
over n squared times, instead
of epsilon, now we have
epsilon plus 1 over n squared.
 
Now, if we take n to
infinity in this
equation, see what happens.
Well, this term here converges
to 0 because n squared is much
bigger than n minus 1.
And this term here converges
to number 1
over epsilon squared.
So it becomes 0 times 1 over
epsilon squared, hence the
whole term converges to 0.
And this proves that indeed, the
limit of the term here as
n going to infinity is equal
to 0, and that implies xn
converges to 0 in probability.
 
Now, there is the one thing
I did not justify in the
process, which is why is
probability of absolute value
xn greater than epsilon less
then the term right here?
So let's take a look.
Well, the easiest way to see
this is to see what ranges of
xn are we talking about
in each case.
Well, in the first case, we're
looking at interval around 0
plus minus epsilon and xn
can lie anywhere here.
 
While in the second case, right
here, we can see that
the set of range values for xn
is precisely this interval
here, which was the same as
before, but now, we actually
have less on this side, where
the starting point and the
interval on the right is
epsilon plus 2 over n.
And therefore, the right hand
style captures strictly less
values of xn than the left
hand side, hence the
inequality is true.
Now, we wonder if we can use
the same trick, Chebyshev
Inequality, to derive the
result for yn as well.
Let's take a look.
The probability of yn minus it's
mean, 1, greater or equal
to epsilon.
From the Chebyshev Inequality,
we have variance of yn divided
by epsilon squared.
Now, there is a problem.
The variance of yn
is very big.
In fact, it is equal
to n minus 1.
And we calculated in part A,
divided by epsilon squared.
And this quantity here diverges
as n going to
infinity to infinity itself.
So in this case, the Chebyshev
Inequality does not tell us
much information of whether
yn converges or not.
Now, going to part C, the
question is although we don't
know anything about yn from just
the Chebyshev Inequality,
does yn converge to
anything at all?
Well, it turns out it does.
In fact, we don't have to
go through anything more
complicated than distribution
yn itself.
So from the distribution yn, we
know that absolute value of
yn greater or equal to epsilon
is equal to 1 over n whenever
epsilon is less than n.
And this is true because we know
yn has a lot of mass at 0
and a tiny bit a mass at value
1 over n at location n.
So if we draw the cutoff here
at epsilon, then the
probability of yn landing to the
right of epsilon is simply
equal to 1 over n.
And this tells us, if we take
the limit of n going to
infinity and measure the
probability that yn--
just to write it clearly--
deviates from 0 by more than
epsilon, this is equal to the
limit as n going to infinity
of 1 over n.
And that is equal to 0.
From this calculation, we know
that yn does converge to 0 in
probability as n going
to infinity.
 
For part D, we'd like to know
whether the convergence in
probability implies the
convergence in expectation.
That is, if we have a sequence
of random variables, let's
call it zn, that converges to
number c in probability as n
going to infinity, does it also
imply that the limit as n
going to infinity of the
expected value of zn also
converges to c.
Is that true?
Well, intuitively it is true,
because in the limit, zn
almost looks like it
concentrates on c solely,
hence we might expect that the
expected value is also going
to c itself.
Well, unfortunately, that
is not quite true.
In fact, we have the proof right
here by looking at yn.
For yn, we know that the
expected value of yn is equal
to 1 for all n.
It does not matter
how big n gets.
But we also know front part C
that yn does converge to 0 in
probability.
And this means somehow, yn can
get very close to 0, yet it's
expected value still
stays one away.
And the reason again, we go
back to the way yn was
constructed.
Now, as n goes to infinity, the
probability of yn being at
0, 1 minus 1 over
n, approaches 1.
So it's very likely that yn is
having a value 0, but whenever
on the off chance that yn takes
a value other than 0,
it's a huge number.
It is n, even though it has a
small probability of 1 over n.
Adding these two factors
together, it tells us the
expected value of yn
always stays at 1.
And however, in probability,
it's very likely
that y is around 0.
So this example tells us
converges in probability is
not that strong.
That tells us something about
the random variables but it
does not tell us whether the
mean value of the random
variables converge to
the same number.
 
