Let's have a look at the geometric
interpretation of the Singular Value
Decomposition. Well, what do we have here?  We have, well to give a geometric
interpretation, it's a little hard to go
beyond R^2, two space in reals.  And
notice that therefore we're going to
look at a matrix that's a 2 by 2 matrix
that takes vectors from R^2 to R^2.
And we've already talked a lot about the
unit ball.  So this here is the unit ball.
And how A transforms vectors can be
completely described by looking how
vectors of unit length are transformed.
And typically if you look at the
projection of the unit ball or the image
of the unit ball, it looks elongated like
this.  And that comes from the fact that
you're taking the linear combinations
and matrix-vector multiply preserves [the
linear transformation] the linear
combination.  Okay, so, in this particular
case, U would consist of u_0, u_1.  Sigma
would consist of sigma_0, sigma_1.  And V
would consist of v_0, v_1, like that.  Now the 2-norm of A would be sigma_0, the largest singular
value.  And notice that that was the
maximum magnification of a unit vector.
This is the maximum magnification. Okay?  So this distance right here is really
sigma_0, from here to here. Alright?  Now in coming up with the Singular
Value Decomposition what did we say?  We said, "Let's pick
v_0 so that it's the unit vector,
unit length vector, such that it achieves
the maximum magnification."  So what does that mean?  That means that over here
there's some vector, v_0, that maps to
this vector right here.  So this here is A
times v_0.  Okay?  Now we then said, "Let's
pick u_0 in that direction but of length
1. So we could put a unit circle here, but
let's just put it right here and say that that is unit length.  And that is u_0.  We then
said let's fill out U with a
orthonormal basis.  So what that meant was
that we found another vector of the unit
length, u_1, which must be perpendicular, orthogonal.
And similarly we filled out v_1 here.
Let's say we pick this one right here. Okay?
And then actually this became our submatrix B and we proved that those had to
be 0.  So this is the geometric
interpretation. If you now take any
vector on the unit circle, that vector
can be written as a linear combination
of these two vectors.  And the vector that
that then maps to is the same linear
combination of this vector and that
vector. As a matter of fact, this
vector right here is the vector sigma 1
times u 1. Okay? Now you can sort of also
see what happens if the smallest
singular value is actually equal to 0.
Then this whole thing would collapse
into a line.  And that's when the matrix
is singular.  Why? Because you wouldn't be able to get
an inverse function, if you started with
any vector that was not on that line.
We're going to see that it is actually
the ratio between this distance and this
distance that determines how close to
singular
the matrix is. Okay?  If this is a perfect
circle, then you actually end up with a
matrix with condition number equal to 1.
Those are the good ones.  Anyway that's
the picture.
