>> Hola, Amigos.
Que tal? Welcome back.
We finally made it to the end of our
introduction to applied linear algebra.
This last sequence of videos is going to focus
on something called the eigenvalue problem.
I would say that we saved one of the
most challenging and rich problems
for the last one that we studied.
This eigenvalue problem is going
to show up all over the place.
You'll see that in this video.
Strictly speaking, when I
say the eigenvalue problem,
it makes it sound like it's a single
problem, but that's not the case.
The eigenvalue problem actually
describes a whole collection of templates
for different types of problems
that arise in the wild.
In linear algebraic theory, we usually
categorized eigenvalue problems
into three different types of problems.
We're going to see all three of
those in this short introduction.
The first one is the least complex and that's
known as the standard eigenvalue problem.
In the standard eigenvalue problem, we're given
a square n by n matrix and then we're searching
for scalars lambda paired with vectors X such
that when I multiply A times the vector X,
I get a scalar multiple of the vector X out.
This scalar is known as an
eigenvalue or a standard eigenvalue
and then this vector is the associated
eigenvector for eigenvalue lambda.
That's the Greek letter lambda.
A second and more advanced
eigenvalue problem that shows
up is something called the
generalized eigenvalue problem.
It starts with the same left-hand
side as the standard value problem.
You have a given modelling Matrix
A. I'm looking for a vector X
such that lambda times B times X
is equal to A times X. The reason
that we call this thing the
general idea value problem
and this the standard eigenvalue
problem is one, this is the standard
from which all other eigenvalue problems arise.
And then two, if we look at how this
standard eigenvalue problem works out,
we see that we could actually put an
identity matrix in between the scalar lambda
and the vector X because I know the
identity matrix times X is just X itself.
The moment that we do that we dropping an
n by n identity matrix in between those.
We might ask ourselves, "Hey, what
if instead of the identity matrix,
that was just any general n by n matrix?"
Well, that is the general eigenvalue problem.
Replace the identity matrix
with a general square matrix
and still maintain the same
relationship that A times X is equal
to lambda times a matrix times X.
In the standard eigenvalue problem,
that matrix is the identity.
In the general eigenvalue
problem, it's the general matrix.
In the development of these theories, once
we've mastered the standard eigenvalue problem
and general eigenvalue problem,
usually we would go
on to master something called
the quadratic eigenvalue problem.
And in the quadratic eigenvalue problem,
we start with modelling matrices M, B,
and K. Each of those is square and shows
up in some real-world modelling context
and then we're looking for eigenvalue
eigenvector pairs in the form lambda
and X such that lambda squared M times X
plus lambda B times X plus KX equals zero.
Claim the transformation
from the general eigenvalue
to the quadratic eigenvalue problem
includes some advanced mathematical theory.
And in fact, in our introduction we're not
even going to touch these later two problem.
I did want to mention these more
general problems because the work we do
to study the standard eigenvalue
problem sets the foundation
for these more advanced techniques.
When solving each of these problem types
that are grouped together and known
as eigenvalue problems, we're going to
focus on two different categories of theory.
For now, let's filter all of our work
through the standard eigenvalue problem
since this is introduction and the focus
of this introduction is going to be
on the most foundational of the three problems.
The good news for us is as we work
to solve standard eigenvalue problem,
we can dip into large volumes of theory that
have been created over the last 350 years.
I'm going to encourage you to think about
that theory as having two categories.
The first category is what I
would call an algebraic approach
to solve each of those problem.
Each of those problems strictly stated are in
algebraic terms, so we can use theorems and pen
and paper analysis to work through each of
the challenges that arise in those problems.
The other approach that is very, very powerful
is something called a numerical approach
to solving these problems, which is instead
of doing our work with pen and paper analysis
and quoting theorems, can we store
or modelling matrices on computers
and then use algorithms software on
those computers to generate eigenvalue
and eigenvector information
associated with those matrices?
At this point in human history,
we've been solving eigenvalue problems
numerically for about 70 years.
1950s was really the first foray
into this after the ENIAC computer.
Many of the best algorithms in the world
that implement solvers to find eigenvalue
and eigenvector information are based
on theoretic and algebraic results
that give insight into these problems.
Because this is an introductory course,
my main goal is to help set the foundation
for your future study and hopefully
help you cultivate some interest
in learning more about this theory.
With that in mind, let's focus in on
a standard eigenvalue problem and talk
where it arises in the world around us.
Here's a verbal description of
the standard eigenvalue problem.
Given a square matrix A that has n rows and
n columns, we want to find all scalars lambda
and associated n by 1 vectors X such
that when I multiply A times the unknown
and desired eigenvector X, I
get the scalar lambda multiplied
by X. Let me call your attention to a few
different features of this algebraic statement.
First, when we say given, we mean we spend days,
weeks, months or years generating that matrix
from some valuable modelling
problem and then the goal is
to find these two separate quantities, a
scalar and a vector, that when paired together,
satisfy the condition that
multiplying the modelling matrix
by the vector gives the scalar
multiple of that vector out.
Some fun intuition that I like to think about
is if I imagined A to be an audio system
and I imagine X to be the vector of
my vocal input to that audio system,
lambda could be the volume
that the audio system puts out.
And in an amplifier, what I hope is
when I put my voice into a microphone,
the audio system amplifies my voice out.
In other words, when I put that
vector into the larger system,
I get a scalar multiple of that vector out.
The system only scales the vector.
It doesn't fundamentally
change the vector underneath.
Now, I'm going to make a claim.
When working with students, I generally
say that my students are geniuses.
They have so much valuable lived experiences
and one of the jobs that I have is
to actually show them they already
know the mathematics that we study.
They just haven't recognized it in a
formal setting like we would in school.
So we're going to play this game have you ever.
Have you ever heard a car rattle when you
turned the music up really, really high?
So really high volume and
you hear the "trrr, trrr."
So the base is going to hear
that rattling sound.
Let's look at a different situation.
Have you ever felt the steering
wheel of your car shake
when you put the brakes on and slow down?
That happens all the time in my Honda Odyssey.
The thing shakes as I'm slowing down.
Have you ever tuned a string
instrument like a guitar or violin?
How about purposely jumped up and down or back
and forth on a footbridge that was hanging
between two points and felt the thing kind of
sway back and forth in a predictable motion?
If you answered yes to any of
these have you ever questions,
then my claim is you've actually
experienced phenomenon in the real world
that can be mathematized
as an eigenvalue problem.
Another way to say that is if you are
willing to spend a few years of your life
and you are super excited about doing this,
you could actually turn each of these problems
into an eigenvalue problem using
formal mathematical technology.
The bad news is that each of these applications,
if I were to actually mathematize those
to state the standard eigenvalue
problem, I would probably need
at least one separate bachelor's degree for each
application and maybe even a master's degree
to do the entire modelling
problem start to finish by myself.
In other words, in an introductory course
where the goal is to introduce ideas,
we have this trade-off between
showing students why this is powerful
and placing reasonable expectations on
students to be able to study an introduction
that sets the foundations
without overwhelming the student
or putting too much burden on their shoulders.
However, I want to avoid the trap of
not mentioning the future applications
and just going right into the theory/.
With this in mind, I want to touch on briefly
some of the much larger application areas
that eigenvalues show up in as a
way to motivate your curiosity.
One of the questions I always
ask myself is why should I care?
How can I use this in my future?
And why does it matter to me?
Let's see if we can help you get answers
to those types of questions by focussing
on other people's answers
to those types of questions.
The first field in which eigenvalue problems
are ubiquitous I would call mechanical
vibrations analysis.
Another way to say this is it's the study
of things that move back and forth or shake.
In other words, the study of
physical objects that vibrate.
Vibrations analysis.
For engineering students who are thinking about
any design of suspension system in any type
of moving vehicle, another word for this idea
of vibrations analysis is
called structural dynamics.
Here's a book by Craig and Kurdila called
the Fundamentals of Structural Dynamics
and what you see on the front here is
this apparatus where you have a real car.
In the background, there's a cute
computer model of that car and the axles
of the car are connected to a suspension system
that is connected to a data acquisition system
that allows the engineers that designed this
to test the response of the design system
against different stimulus on the road.
Another way to say this is suppose I
have like a mass on top of some springs.
There's also some fluid inside the
springs like a dampener and on the bottom
of that system we've got this
wheel and the mass and the wheel go
over these bumps called speed
bumps or potholes or whatever
and the question is how does the
displacement of that wheel show up over time?
So if I hit the thing up and the spring
compress, what does that look like?
Do you see stuff moving back and forth?
Yep. That thing is going to go up and down
and the moment that it goes up and down,
we're thinking eigenvalue problem.
That's a vibration problem.
Now, the funny thing about this problem is it
has this analog that's going to show up all
over the place, and, in fact, we're
going to study this quite deeply
because it's the foundations
upon which this problem rests.
That analog is called a mass spring system.
So in this case, you got to think like,
OK, the mass is sitting on top of springs
and then I'm literally like changing the
way that the mass is related to the spring
and the question is how is
the displacement happen?
Well, that's what's happening over here.
That vibration problem is a lowercase example.
It's not as complex as the
suspension system example.
Another situation in which this arises
is in the brake system in any vehicle.
That's exactly the phenomenon
that I was talking about.
When you push the break in your car and
you feel that rattle, that's a vibration.
When you see vibrations,
think yourself, "Oh, my God.
There's probably an eigenvalue
problem underneath this."
Another example of where eigenvalues show
up in mechanical vibrations
analysis is analyzing the motion
of buildings during earthquakes.
I do want to give a call out that all of the
diagrams that I get either came from this book
by Craig and Kurdila or this book
on Mechanical Vibrations by S. Roa.
Don't run and go get these books immediately
if you go take upper-division
classes in these theories.
I actually get a lot of information
out of these books,
but the diagrams that you see here
come from both of those books.
Remember, the claim is that vibrations problems,
physical objects that shake back and forth
under different phenomenon, those
give rise to eigenvalue problems.
Check this out.
If I have a 4-story building where each story
has steel beams and then this large mass,
which is called the floor itself, and
the beams are called the structure
and then I shake the bottom of that thing, what
is the corresponding motion of that building?
That problem is at least the
fourth year of an undergraduate
and more like a master's or PhD problem.
There have been multiple generations
of PhD and postdoctoral researchers
who have studied the theory behind this problem
and then there's entire engineering firms
that hire very well-educated people to do this
type of analysis when designing buildings.
But if we're thinking about
introductions, we can map this larger problem
down to a much simpler problem, which
is if we have this mass attached
to two springs, hey, mass and springs.
Are you recognizing a pattern?
Mass and springs.
And then that mass has this little
joint connected to another mass
and then we shake the mass on the bottom,
so we literally like perturb it back
and forth, AKA, an earthquake happens.
What happens to the mass sitting up
here given the stiffness of that joint?
We can use observation and intuition that
we build about this simplified problem
to analyze the much more complex problem, which
is how do buildings shake under earthquakes?
If we wanted to up the ante even more, we could
also talk about the motion of bridges in wind.
This is a super-complex problem
and requires a ton of education
to even understand the statement of the problem.
I want to give a shout-out to engineers
that have to take a course called statics.
So on the front of statics books, you
often see like these bridges, and, in fact,
later on there's something
known as a truss system.
T-r-u-s-s.
Here are a bunch of different truss systems
corresponding to designs of bridges and cranes.
In statics, you look at the response
of a particular system under
a static non-moving force.
In other words, this is all algebraic
analysis dedicated to systems
with many, many different components.
Generally, just as we started our
mathematical analysis with algebra and then went
into calculus, statics sets the foundation for
a more complex theory, which is called dynamics.
So in the analysis of the motion of bridges,
you would take a statics class to figure
out what truss systems is, a dynamics class to
figure out how things move and how they respond
to time varying forces, and then you would take
a number of specialized courses in the materials
and construction of civil structures.
That's what civil engineers do.
I did want to give a shout-out and
just show in every dynamics book
and even statics book we'll always have
these conversations about vibrations
and time responses, and, in fact, here
you see the suspension system of a car
and there's a nice diagram
associated with that and it is really,
really funny that almost
every single time the places
where all these conversations
start is spring-mass systems.
Just in case my aeronautics and aerospace
engineering friends were feeling left out.
Don't trip.
Guess what?
These vibrations analysis problems
show up in the dynamics of motion
of the wings of planes in flight.
In a rudimentary introductory analysis, we
can think about a plane as having a mass
and then two long wings attached to that.
That's the fuselage and the
wing from a front view.
But then when I think about this,
those things could be point masses.
We could model those as point masses at the
end of the wing and then the mass in the middle
and then springs connecting each of those
long rods to the actual mass so that
as the thing goes up and
down, you see this kind of up
and down motion, this vibration of the wings.
And the question that we ask ourselves
is, well, how does the different forces
that the wings experience relate
to the up and down motion?
And have I designed my system in such a way
that the wings aren't going
to fall off when in flight?
Once we have intuition about
that introductory problem,
we can actually make it much more
interesting by discretizing a larger wing.
So there's the mass and then instead of
thinking about a single mass on the outside,
we can think about like 12 separate masses.
We can kind of lump together the
different elements along the wing
and discretize those masses.
And then the question is as
we move the plane up and down,
what are the different forces experienced?
And how does the actual motion of
the wing relate to those forces?
Electrical and computer engineers,
were you feeling left out?
Don't trip.
Eigenvalues show up in electrical
circuit analysis and design
and specifically there's something
called resonance in circuits.
This is part of resistors,
capacitors, and inductors.
Linear circuit analysis.
That sets the foundation for
computer-aided design in circuits and many
of the other techniques that
engineers use when designing circuits.
Eigenvalues show up as a
fundamental tool in that analysis.
For computer and signal processing engineers,
this also shows up in computer
image compression algorithms.
Eigenvalues can be used to compress large
amounts of data into smaller amounts of data.
In fact, in recent fields known as
machine learning and data analysis,
this is all the rage in many
of the technical fields.
Many techniques in those fields are based on
this concept of principal component analysis.
Principal component analysis is at
its heart and eigenvalue algorithm.
And the hits just keep on coming.
There something called Google's
PageRank algorithm.
This is the fundamental technology
that Google used to get off the ground.
There's a great book on this by authors
Langville and Meyer called Google's PageRank
and Beyond: The Science of Search Engine.
It goes very deep into the
foundations of Google's technology.
And I just wanted to show you that in the
computation of Google's PageRank Vector,
they solve this eigenvector
problem, which they state like this.
This is a constrained minimization problem.
Don't worry about that.
That's called Rayleigh quotient.
That's beyond the context
of an introductory course.
But the point of the matter is
without eigenvectors and eigenvalues,
Google would not exist in the
form that it currently does.
Similarly, Netflix's old
prediction algorithms of ratings.
Have you ever been watching a Netflix show
and they're like, oh, we 98 percent predict
that you're going to like this movie
and then you click on it because they're
like telling you what you
like and what you don't like?
Their old algorithm was based on
prediction capacity using eigenvalue theory.
Moreover, when we're thinking
about the structures behind how
to turn real world modelling problems into
eigenvalue problems, many of the ideas that show
up in our mathematical modelling of mechanical
vibrations actually set the foundation
for a much larger field called
finite element analysis.
And just to show you how powerful this is,
many of Boeing's planes starting the 1980s,
the engineer that designed those planes,
the teams of engineers actually used finite
element analysis to design the entire structures
of the plane prior to getting
the first model built.
There's this famous workflow problem where we
go from the mathematical model of the plane,
the mathematical analysis that
arises using finite element methods,
to building a small version of the plane.
I think that's called a string.
So here, this version is actually
like smaller than the size of a human.
It's connected to all these sensors and
then there's these wires that come down
and this small version will fly under similar
conditions that it would see in the real world,
so they have like these wind tunnels
and they expose it to all these forces
and the question is using the
information they collect from these data
and the mathematical analysis that they have in
finite element method, can they iterate enough
to produce a plane that works well and that
they can certify that the billion dollars
of investment that it takes
to build will not be a waste?
Finite element methods also show up
in the analysis of wind turbines.
Could you imagine what vibrations
happen when you're having a large set
of blades move around in the wind?
You'd have vibrations of the actual blades.
You'd have vibration of the foundation.
And you see what's happening here is where
discretizing the entire metal structure
into these small points and then edges.
The points are masses.
The edges are like springs.
And then the question is as we expose this thing
to a bunch of different forces, what happens?
That brings us into a bunch of applications
having to do with health analysis.
So there's finite element
analysis of cuts on a human hand
and the corresponding sutures
that come into that.
Movement of the human heart
and how blood flows in and out.
The shearing and compression forces
experienced on a car in accidents.
Weather models for pressure and precipitation.
Models involving rockets and rocket science.
So here we have a bunch of
masses and a single beam.
Here we start putting the shell outside
and then we compose them together,
which means we can do analysis
on each individual component,
fit them together and do analysis like that.
Models of the frame of a school bus or any
bus that moves people around and round.
The list just keeps on going and what I want
to show you is underneath all these crazy
models are mass spring system literally
in the same form that we've been studying
in this class where you have a matrix K,
a right-hand side vector F, and then a vector
of displacement U along each point mass.
Is your jaw on the floor yet?
Let me just go ahead and
close that mouth for a second.
The eigenvalue problem sets the foundation
for so many problems in applied mathematics
and engineering that this
problem is famous for math heads.
The goal of an introduction is not
to get into the advanced theory.
That's the purpose of a bachelor's,
master's and PhD program.
Instead, what we do is we set the foundation
upon which the advanced theory can be built.
So in order to start setting that foundation,
I want you to go back and look at the list
of applications that we just covered and I
want to show you two themes that show up.
Number one, the first theme is that many
of those problems are continuous problems.
What I mean is they exist in time and space.
The way that I would say that as
a mathematician is those problems,
the analysis of brake systems, the analysis of
train motion or rockets or planes or bridges
or buildings, those involve
math that are built on calculus.
In other words, the equations that govern that
have continuous functions in the equation.
That is beyond the scope of algebra.
That's not working with equation
that involves scale
or valued variables or algebraic equations.
That's working with differential equations,
equations that involve functions
and derivatives.
Thinking back to our own time
studying algebra and calculus,
remember that calculus has a lot more
going on because time is difficult.
It's so much easier to analyze stuff
that is static than stuff that changes,
which implies is there a way that
we can use our knowledge of algebra
to say something meaningful in calculus?
That's exactly what eigenvalues do.
They turn continuous problems
into discrete problems.
They turn calculus problems
into algebra problems.
The second theme that shows up is that
many of those problems involve systems
with a large number of dimensions.
Now, large is relative.
If I've only done mathematical analysis on
the system with one dimension and then I go
to two dimensions, that's much, much larger.
But what you'll see is as we
increase the complexity of our model,
more and more components are
included in the models that we use
to get closer and closer to real life.
From that context, eigenvalue problems
and eigenvalue theory is often used
to reduce large number of dimensions
to a much smaller set of key dimensions
that still replicate the fundamental behavior
of our larger data using a model
with much smaller dimension.
Because this video is not meant to be all
inclusive and because I know that many
of my viewers are super curious and want to
know more, I want to show you how you can find
out more from other YouTube content creators.
If you go to my website
appliedlinearalgebra.com, go over to Blog,
scroll down to Math 2B: Linear Algebra, and
then scroll all the way down to Lesson 27,
that's my first eigenvalue lesson.
If you see the eigenvalue extras
playlist, go ahead and click on that.
This playlist is a collection of videos
that I found over the last few years
where different content creators
are spending a lot
of their time putting together what I
would call relatively strong stories
about eigenvalues in the real world.
I'm not going to test on this in my class, but
I do want to give you access to those resources
so that if your curious mind seeks answers
to questions like, wait, I want to learn more
about that idea, these are some
places that you can find answers
to that outside of my learning resources.
I hope that list of applications sparked
some curiosity and got you thinking
about your own academic interests and how you
might leverage this technology in your future.
For us in this introduction,
my goal is not to get advanced.
My goal is to set a very solid foundation.
So when we study the standard eigenvalue
problem, which is given a modelling matrix
that is n by n square, find a
lambda scalar and a vector X
that when paired together satisfy A times
X equals lambda times X. We're going
to motivate this problem by studying what I
call a keystone application that gives rise
to an authentic eigenvalue problem that
you're going to study using your eyes,
hands and actually feel in our course.
Now, you might not understand what
I mean by keystone application.
Let me talk about that.
I like to think about keystone applications
like I think about the keystone in a Roman arch.
If you've never heard of that idea, go ahead and
search the term keystone on Google or DuckDuckGo
and then go to images and you'll see these
beautiful images of these arches made of stone
and at the very, very top there's this keystone,
and, in fact, there's a beautiful Wikipedia page
on keystones in architecture that talks
about the keystone as this center stone,
so if you didn't have that keystone in the
actual arch, the entire thing would fall
down because the stones themselves are cut in
such a way that all of the stones kind of line
up and then when you put this
Keystone in, the arch supports itself.
It won't fall over and in that way the
keystone allow this to be a gateway,
allow this arch to stand up and
let stuff to pass through it.
Similarly, when I'm coaching young students, I
look for keystone applications that will hold
up our theory and can be used as central
components for future gateway studies.
And so what I mean by that is once you have
this foundation, you can pass through a bunch
of new information and the arch that you
built in your introduction won't fall down.
You have a very, very solid understanding
that's based on these critical ideas.
So from that standpoint, in our introduction,
I'm going to actually give you a
keystone problem which is a version
of that spring-mass problem central to
so many of the idea that we just covered.
Not only that, but I'm actually going
to guide you through the process
of mathematizing our problem and then using
fundamental eigenvalue theory to solve
that problem and give useful
information about the dynamics underneath.
With that in mind, my objectives for
our eigenvalue theory are the following.
Number one, I'm going to help you build your
own eigenvalue problems from a phenomenon
that you observe in your real world.
Number two, I'm going to help you learn theory
to solve this problem using introductory
techniques and computer tools.
Number three, I want to actively
help you plan for
and describe more advanced eigenvalue
problems that you might want to study
in your future academic and
career pursuits and the hope is
that this introduction makes
that future study much easier.
At the end of my work with students,
I always ask them to grade me
on my effectiveness in accomplishing
these goals.
I would challenge you to do the same thing.
In the next video, let's hop into the
keystone application and get a sense of how
that application works as a way
to motivate eigenvalue theory.
I'll see you there.
