[MUSIC]
Hopefully, you will all now
have a reasonable feeling for
what an eigen-problem looks like geometrically.
So in this video, we're going to formalise
this concept into an algebraic expression,
which will allow us to
calculate eigenvalues and
eigenvectors whenever they exist.
Once you've understood this method,
we'll be in a good position to see
why you should be glad that
computers can do this for you.
If we consider a transformation A,
what we have seen is that if it has
eigenvectors at all, then these are simply
the vectors which stay on the same
span following a transformation.
They can change length and even point
in an opposite direction entirely.
But if they remain in the same span,
they are eigenvectors.
If we call our eigenvector x,
then we can say the following expression.
Ax = lambda x.
Where, on the left hand side, we're applying
the transformation matrix A to a vector x.
And on the right-hand side,
we are simply stretching a factor
x by some scalar factor lambda.
So lambda is just some number.
We're trying to find values of
x that make the two sides equal.
Another way of saying this is that for
our eigenvectors,
having A apply to them just scales
their length or does nothing at all,
which is the same as scaling
the length by a factor of 1.
So in this equation,
A is an n dimensional transform,
meaning it must be an n
by n square matrix.
The eigenvector x must therefore
be an n-dimensional vector.
To help us find the solutions
to this expression,
we can rewrite it by putting all the terms
on one side and then factorizing.
So 
(A - lambda I) x = 0.
If you're wondering where
the I term came from,
it's just an n by n identity matrix, which
means it's a matrix the same size as A but
with ones along the leading diagonal and
zeros everywhere else.
We didn't need this in
the first expression we wrote,
as multiplying vectors
by scalars is defined.
However, subtracting scalars
from matrices is not defined, so
the I just tidies up the maths,
without changing the meaning.
Now that we have this expression we
can see that for the left-hand side to
equal 0, either the contents of the
brackets must be 0 or the vector x is 0.
So we're actually not interested in
the case where the vector x is 0.
That's when it has no length or direction
and is what we call a trivial solution.
Instead, we must find when
the term in the brackets is 0.
Referring back to the material in
the previous parts of the course, we can
test if a matrix operation will result in
a 0 output by calculating its determinant.
So, 
det (A - lambda I) = 0.
Calculating the determinants manually is a
lot of work for high dimensional matrices.
So let's just try applying this to
an arbitrary two by two transformation.
Let's say, A = 
(a, b, c, d).
Substituting this into our eigen-finding
expression gives the following.
Det (a b c d)-
lambda 0 0 lambda.
Evaluating this determinant,
we get what is referred to as
the characteristic polynomial,
which looks like this.
So lambda squared
-(a + d) lambda
+ ad - bc = 0.
Our eigenvalues are simply the solutions
of this equation, and we can then plug
these eigenvalues back into the original
expression to calculate our eigenvectors.
Rather than continuing with our
generalized form, this is a good moment to
apply this to a simple transformation, for
which we already know the eigensolution.
Let's take the case of a vertical
scaling by a factor of two,
which is represented by
the transformation matrix A = 1, 0, 0, 2.
We can then apply the method
that we just described and
take the determinant of A minus lambda
I and then set it to zero and solve.
So det (1- lambda,
0, 0, 2- lambda)
= 1- lambda, 2- lambda,
which is of course equal to 0.
This means that our equation must
have solutions at lambda equals 1 and
lambda equals 2.
Thinking back to our
original eigen-finding
formula, (A - lambda I) x = 0.
We can now sub these
two solutions back in.
So thinking about the case
where lambda = 1,
we can say (1- 1, 0, 0, 2- 1) 
times this x vector,
x1 and x2, must equal to 
(0, 0, 0, 1) x1, x2,
therefore we've got 0 and x2 must equal 0.
Now, thinking about the case
where lamda equals 2,
at lamda = 2, you get 1- 2, and 2- 2,
And then you get of course minus 1, 0, 0, 0,
Which equals to minus x1, 0,
which equals zero.
So what do these two expressions tell us?
Well, in the case where our
eigenvalue lambda equals one,
we've got an eigenvector where
the x2 term must be zero.
But we don't really know
anything about the x1 term.
Well, this is because,
of course any vector that points along
the horizontal axis could be
an eigenvector of this system.
So we write that by saying @ lambda = 1,
x, our eigenvector, can equal anything
along the horizontal axis,
as long as it's 0 in
the vertical direction.
So we put in an arbitrary parameter t.
Similarly for the lambda = 2 case,
We can say that our
eigenvector must equal 0,t.
Because as long as it doesn't move
at all in the horizontal direction,
any vector that's purely
vertical would therefore also
be an eigenvector of this system,
as they all would lie along the same span.
So now we have two eigenvalues, and
their two corresponding eigenvectors.
Let's now try the case of a rotation by
90-degrees anti-clockwise, to ensure that
we get the result that we expect which, if
you remember, is no eigenvectors at all.
The transformation matrix corresponding
to a 90-degree rotation is as follows.
A = (0, -1), (1, 0).
So applying the formula once again we
get the det (0- lambda- 1),
(1, 0- lambda),
which if you calculate this through,
comes out to lambda squared + 1 = 0.
Which doesn't have any real
numbered solutions at all.
Hence, no real eigenvectors.
We can still calculate some complex
eigenvectors using imaginary numbers, but
this is beyond what we need for
this particular course.
Despite all the fun that we've just been
having, the truth is that you will almost
certainly never have to perform
this calculation by hand.
Furthermore, we saw that our approach
required finding the roots of
a polynomial of order n, i.e.,
the dimension of your matrix.
Which means that the problem
will very quickly stop being
possible by analytical methods alone.
So when a computer finds
the eigensolutions of
a 100 dimensional problem it's forced
to employ iterative numerical methods.
However, I can assure you that developing
a strong conceptual understanding of
eigen problems will be much more useful
than being really good at calculating them
by hand.
In this video, we translated our
geometrical understanding of eigenvectors
into a robust mathematical expression,
and validated it on a few test cases.
But I hope that I've also convinced
you that working through lots of eigen-problems,
as is often done in engineering
undergraduate degrees, is not a good
investment of your time if you already
understand the underlying concepts.
This is what computers are for.
Next video, we'll be referring back
to the concept of basis change to
see what magic happens when you
use eigenvectors as your basis.
See you then.
[MUSIC]
