In a linear algebra course, Matrix inverses
are something that get a lot of attention.
There are all these formulas thrown at you
like for the determinants, the inverses of
a 2x2 matrices, cramner’s rule etc etc.
These formulas are great, but I think they
can obscure the very simple idea behind what
an inverse actually is, and when it exists.
You’ve probably been taught that the inverse
exists if and only if the determinant isn’t
0. But most students have no idea what the
determinant has to do with anything let alone
inverses. I’m going to teach a completely
different condition for when an inverse exists,
that I think is much more intuitive.
But before all that:
Do you know what a linear transform is? If
not, go watch my last video because none of
this one is going to make sense at all otherwise.
In fact, if you want to brush up on vectors
and bases, you can watch the video before
that too.
Even though this video is about matrix inverses,
I’m not going to define what an inverse
is straight away. Instead I’m going to define
something called a left inverse because I
think understanding these first will give
you a better intuition for inverses themselves.
Imagine you have a bunch of vectors in some
vector space, in this case in 2d, and you
apply some Matrix M on them, here it rotates
them. Then you think you’d like to undo
what you just did and get the vectors back
to where they were. In this example, what
transformation undoes the rotation? Here,
you’d just rotate everything back by the
same angle. That’s all a left inverse is.
It’s the matrix that undoes the original
matrix, so it’s like you’ve done nothing
at all. If I wanted to write this as an equation,
it’d say, if you do M, then you do L , that’s
the same as if you did nothing. This thing
is called the identity matrix and it just
means the transformation where you do nothing.
This thing is called the left inverse for…
hopefully obvious reasons. So now we know,
the left inverse is a matrix that undoes the
original matrix’s action.
The annoying thing about inverses is really
their name. It sounds like the *inverse* should
be the thing that undoes a matrix. Instead
the definition of M inverse is:
M inverse undoes M
AND
M undoes M inverse.
Going back to our example: if you do M first
and then L, that’s the identity. But it’s
also true that If you did L first, then M,
you’d also get the identity. So since L
undoes M and is undone by M, L and M are inverses
of each other. You might wonder, is the left
inverse always also the inverse like this?
No, obviously not, or they wouldn’t have
different names, would they?
Before we move on, let me ask you a question
to check you’ve understood this so far.
Imagine you have a matrix like this. What
it does is that it takes a 3D vector, and
jumbles up the components. Does this matrix
have a left inverse? As in, can you undo this?
Then if it does have a left inverse, figure
out if it has an inverse as well. Put you
answer in the poll in the corner, and pause
the video now to think about it.
The answer is that it does have a left inverse.
It’s the one that takes a vector like this
and rearranges the components like this. It’s
clear that this is a left inverse of A since
it undoes it like so. But A is also a left
inverse of it, as you can see, because A undoes
this matrix. So B is the inverse of A.
Now that we know what an inverse is, let’s
think more about when they exist or don’t.
Again, it’s going to be more convenient
to look at when a left inverse exists first.
Here’s another question. This matrix takes
a 2d vector a b and sends it to a 0. Does
this matrix have a left inverse? If so, figure
out what it is.
Again put your answer in the poll in the corner
and pause now to think about it.
Notice something about this matrix. It takes
the vector a b to a 0, but it also takes a
d to a 0 as well. This… is a bad thing,
and it’s because of this that the left inverse
doesn’t exist.
Why? Well, say you have some vector v and
M takes it to w. The left inverse of M, if
it exists, knows M and what w is, but it doesn’t
know what vector produced w. Just using the
information given, it needs to find what the
original vector was, so that it can take w
back to where it came from. However. If there’s
some other vector u that also goes to w, the
left inverse has a problem. it can’t just
look at w and know for sure it whether it
came from v or from u because there isn’t
enough information. This means the left inverse
*can’t* take w back to where it came from,
so… it doesn’t exist.
This thing here, where two different vectors
v and u get mapped to the same vector, i.e
M(u)=M(v), is what I’ll call M losing information.
What we’ve just seen is that if M loses
information it doesn’t have a left inverse.
But what about the other way around? If M
doesn’t lose information, does this mean
the left inverse exists? Well, yes actually.
Because all the left inverse has to do to
undo M is find the vector w came from. Since
there’s only one vector v it could be, there
is an inverse that takes w and returns v.
This doesn’t mean it’s easy to find out
what v is necessarily, but looking at w does
in principle give you enough information to
undo M and return v.
So a matrix has a left inverse if and only
if it doesn’t lose information.
Let’s look at another example to understand
this point better. Imagine I have a matrix
from 2d to 3d and what it does is, it rotates
any 2D vector into 3D space like this. Does
this matrix have a left inverse? Pause the
video to think about it.
The answer is, it does have a left inverse
because A doesn’t lose information. If you
want to take vectors like this back, you know
where they came from so all you have to do
is rotate the plane back. Let B be a matrix
that takes 3D vectors to 2D that rotates this
plane back. It is a left inverse of A.
Now, is B the inverse of A? In other words,
Is A B’s left inverse? Pause the video and
think about it.
The answer is, no, B has no left inverse
We’ll show that by showing B loses information.
First, pick any 3D vector that’s not on
this plane. B has to send it to some 2D vector,
so let’s just say here. But there’s another
3D vector that’s already sent there. It’s
this vector u that’s on the plane. So B(u)
is equal to B(v). Hence B loses information
and doesn’t have a left inverse.
There’s an important lesson to be drawn
from this example. You might have wondered
before why we only ever talk about the inverses
of a square matrices. What’s so special
about transformations from n dimensions to
n dimensions? The reason is, non square matrices,
i.e ones from n dimension to m dimensions
never have an inverse. The issue is, if you
have any transformation going from a bigger
space to a smaller space, like B which went
from 3d to 2d, you have to lose information.
These types of matrices always send some vectors
to the same place.
If you have a matrix from a smaller to a bigger
dimension, it’s possible that it has a left
inverse- like A did in our example. But it’s
left inverse goes from big to small, like
B, and so it can’t be undone. Hence, even
though some nonsquare matrices have left inverses,
they never have an inverse.
Square matrices don’t have these issues
at all though. Actually, for square matrices,
everything massively simplifies because the
left inverse is always equal to the inverse.
So if a square matrix has a left inverse,
it automatically has an inverse.
I am not going to prove this fact. You are,
for homework. But I will give you an illustrative
example in a little to help you understand
why it’s true.
Once you’ve proved it, you’ll see that
for a square matrix A, if B undoes A, then
A undoes B as well. This gives us an easy
criteria for checking whether A has an inverse
or not, because it’s the same criteria we
used to check whether A has a left inverse:
Just ask, does A lose information? If yes…
then sorry, A inverse doesn’t exist.
If no, then A inverse does exist.
But you might be thinking, ok, so what? How
is this easy to check? Wouldn’t you have
to compute the outcome for every single vector
that goes into M and compare the results to
every other vector’s and see if any of them
match? Isn’t that beyond tedious?
Thankfully there is an easy way to check this
condition. All you need to do is figure out
which vectors get mapped to 0. For any linear
map, 0 is always mapped to 0, but all you
need to check is if there’s any *other*
vector mapped to 0 or not, and that’s enough
to decide if M loses information. Why would
that be enough?
Imagine two vectors u and v do both go to
w, so M loses information. Then the vector
u-v, by linearity, gets mapped to 0. So whenever
you have 2 vectors going to the same thing
like this, you always get at least 2 vectors
ending up at 0. So you can check whether a
matrix loses information by checking how many
things go to zero. In other words, figure
out how many vectors v satisfy the equation
Mv=0. This is part of why you spend so much
time in linear algebra courses studying the
solutions to equations like this. You can
solve for v by a) using subsitution b) using
Gaussian elimination or (c) by getting your
computer to do the Gaussian elimination for
you.
But the point is, you can find out if M loses
information easily enough this way, and that
tells you whether M has an inverse.
Let me summarise quickly what we learnt in
this video:
1. A left inverse of a matrix is matrix that
undoes it.
2. That the left inverse exists if you don’t
lose information: i.e, if the matrix never
sends two different vectors to the same vector.
3. The inverse is the matrix that both undoes
the matrix, and is undone by the matrix
4. Nonsquare matrices never have inverses.
5. For a square matrix, the left inverse is
equal to the inverse
6. You only need to check if the square matrix
loses information or not to decide if it has
an inverse.
7. You can check whether the matrix loses
information by looking at how many different
vectors get mapped to 0. This you can do by
solving the equation Mv=0.
And so that’s it. But before you run off,
here’s some homework for you. The first
one is multiple choice. Which of these is
the inverse of this matrix? I know that you
can just check each of these to see which
works, but I’d rather you did it another
way. And for crying out loud, don’t use
the formula for the inverse. Once you’ve
figured it out, put your answer in the poll.
Question 2. Prove that for a square matrix
M inverse is equal to the left inverse of
M. There’s lots of hints in the description
for this, but first I want you to try question
3 because it’s a very illustrative example.
Question 3. Suppose we have the matrix from
before. If we have a left inverse for it,
L, then we know L undoes M. We also know M
takes the basis vectors to these vectors,
so L must take them back. First show that
these two new vectors form a basis.
Then, to show that M undoes L, we need to
show that for any vector v, if you apply L
then M, you get v back. Show this by writing
v as a linear combination of the basis in
the first part. Hopefully doing this first
will help you with the proof in question 2.
As you will have noticed, the first question
was from Brilliant.org. What I like about
questions from there it is that they don’t
just give you a formula and then ask questions
where you plug numbers into that formula.
That’s what I found a lot of highschool
and early university textbooks, and it’s
annoying because that doesn’t teach you
anything. Instead, they get you to do examples
like this one, and understand the principle
yourself, then in the next few question, lead
you to finding the general solution on your
own.
As you can see, this is exactly how I like
to learn new maths- before reading the proof
of anything I’ll do lots and lots of simple
examples first and try and understand the
underlying reasoning, so I love how Brilliant
allows you to do this in a structured way.
They have loads of different maths and science
course on their website, which you can access
completely with a monthly or yearly membership.
If you follow the link in the description
or on screen, you can get 20% off an annual
subscription.
Alright, so that’s all for this video, I
hope you enjoyed it. The next one is about
changing basis, which is key to understanding
loads of Quantum mechanics, including the
Heisenberg uncertainty principle. It should
be up here in 2 weeks, as long as I don’t
decide I hate the just after uploading it
like I did for the original version of this
one. Anyway, subscribe if you’d like to
be notified when my next one is up. Thanks
for watching!
