In the last video we worked out the
theory of simultaneous diagonalization,
in the case that one of the matrices
was non-degenerate. That is to say
when one of the matrices had all of its
eigenvalues with multiplicity one.
Now we're going to tackle the general
case. So the theorem we're going
to prove is that if you have two
diagonalizable matrices
or two diagonalizable operators and
they commute, then they're simultaneously
diagonalizable. Now the first step is to
show how B acts on the eigenspaces
of A. So let's suppose that E lambda is
an eigenspace of A corresponding to
the eigenvalue lambda. Then I claim
that if v is in that eigenspace,
then so is Bv. In other words, B maps
the lambda eigenspace to itself.
This is always assuming they commute.
Okay, so here's proof.
So we know that Bv is lambda v and
we look - I'm sorry, that Av is lambda v -
and we want to ask what is A times 
Bv. Well that's AB times v
and that's BA times v. And A times v
is lambda v and you pull the lambda
out in front and you get lambda times
Bv. So Bv is also an eigenvector
with eigenvalue lambda,
that's our lemma.
Okay, now armed with that lemma,
here is the proof of the theorem.
So first we're going to pick a basis that
diagonalizes A.
Now some of the eigenvalues of A
might be repeated. You might have
lambda one a bunch of times and then
lambda two a bunch of times,
and all the way out to lambda k.
That's fine. And we've just shown that
B in the basis, since it sends all
of these vectors to linear combinations
of themselves, that gives you a block
up here, and it sends all the things in
this eigenspace to itself,
gives you a block here. So it makes B into
a block diagonal matrix.
And then we diagonalize each block.
And if you diagonalize a block,
that gives you an eigenvector of this
whole matrix. You take the eigenvector
of B1 and you pad it with zeros or the
eigenvector of B2 and you pad it above
with zeros and below with zeros.
So, now the eigenvectors of the block
are also eigenvectors of this block
because this block is just lambda times
the identity. Everything's an eigenvector
of the identity. So now we've got
vectors that are eigenvectors
of this block and this block. 
Eigenvectors of this block and this block.
And they correspond to eigenvectors of
both A and B, and we're done.
Now that may have gone by a bit fast,
so let's go over this in the case
of a specific example and see how
it works. So here are two matrices,
they're both diagonalizable and they
commute. If you multiple them in either
order, you get this matrix. Don't believe
me? Work it out yourself.
So we want to find a set of vectors that
are eigenvectors of both A and B.
The first step is to diagonalize A and
diagonalizing A isn't hard
because A is a block matrix.
We diagonalize this corner.
We diagonalize this corner. And what we
discover is that there are
two eigenvalues, 2 and lambda 2, and that
each one has multiplicity two.
You get an eigenvector 2, 1 from the
upper left corner and 0, 0, 2, 1
from the lower right corner. Likewise for
eigenvalue -2, you get 2, -1 from the
upper left, 2, -1 from the lower right.
So here are our four eigenvectors
of A, two of them with eigenvalue 2,
two of them with eigenvalue -2.
And I'm calling these bases, the D
basis because I'm saving the letter B
for our final answer. In other words,
A in the D basis is block diagonal,
lambda 1, lambda 1, lambda 2,
lambda 2.
The next step is we need to figure out
what is B in that basis,
so we compute. We compute what is
B times D1. We multiply the matrix B
times D1 and you discover that you
get D2. Well it has it be something
in that - a linear combination of D1
and D2 because it has to be
an eigenvector of A, with eigenvalue 2.
And it turns out to be exactly D2.
Likewise, D multiplied by D2 gives you
exactly D1. B times D3 gives you D4.
B times D4 gives you D3. So that means
that the matrix of B in the D basis
takes this form, which again, it's blocks.
So this is our block B1 and this
is our block B2, and it corresponds to
this twice the identity and
minus twice the identity.
Okay, so the next step is to diagonalize
each block. The upper left block was just
0, 1, 1, 0 and we know what the
eigenvalues of that are,
it's 1 and -1, and the eigenvectors are
1, 1 and 1, -1.
And the lower right block happened to
look exactly the same, coincidentally,
so same answer. Great.
Now we can diagonalize the whole
matrix B in the D basis.
If you have the eigenvectors of this
corner, you pad it with zeros
in the bottom and you get an
eigenvector of the whole matrix.
If you have an eigenvector of this
corner, you pad it with zeros
at the top and you get an eigenvector
of the whole matrix.
Remember our tricks of the trade,
blocked diagonal matrices.
So here we go. We take the first two
eigenvectors, pad them with zeros.
Second two eigenvectors, pad them at
the top with zeros.
So now we've got our four eigenvectors
of B in the D basis,
with eigenvalues 1 and -1, and 1 and -1.
Okay, now we want to convert back.
We want to say, "Hey, 1, 1, 0, 0
in the D basis really means 1 times
D1 plus 1 times D2."
That's 2, 1, 2, 1.
1, -1 in the D basis means
D1 minus D2, 2, 1, -2, -1.
0, 0, 1, 1 in the D basis means
D3 plus D4.
This means D3 minus D4.
So these correspond to the original
vectors, this, this, this, and this.
And I claim that we're now done, that these
are eigenvectors of A and they're
eigenvectors of B. They're eigenvectors
of A with eigenvalues 2, 2, -2, and -2.
They're eigenvectors of B with
eigenvalues 1, -1, 1, and -1.
Now if you want to check, in other
words, A and B in the B basis
take these forms. And if you want to
check, you just multiply it out.
A times B1 here was A, here's B1,
you multiply it out and sure enough
it's twice B1. B times B1, you multiple it
out, it's just B1.
So B1 really is an eigenvector of A and
an eigenvector of B.
And you can check for B2, B3, and B4.
