The following content is
provided under a Creative
Commons license.
Your support will help
MIT OpenCourseWare
continue to offer high-quality
educational resources for free.
To make a donation or
view additional materials
from hundreds of MIT courses,
visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: All right.
So last time we
talked about methods
for solving recurrences,
and we spent
most of our time talking about
divide-and-conquer recurrences.
These are recurrences where
you break the problem up
into much smaller sub-problems,
like half the size
or 2/3 the size.
And they come up quite a
bit in computer science
when you're doing algorithm
design and algorithm analysis.
Today, we're going
to spend our time
talking about a different
kind of recurrence that's
called a linear recurrence.
They also come up
in computer science
and a lot of other fields.
Now, I'm going to give
you the formal definition
of a linear recurrence later.
First I want to start
with an example,
and this is an example of a
linear recurrence that comes up
in population modeling.
In fact, it comes up
in a lot of places.
And to start it off, we're going
to analyze a particular problem
that we call the
graduate student job
problem or the graduate
student job nightmare.
And the question here
is, will your TAs
be able to get a job as
a professor somewhere
when they get their PhD?
So this is a problem
they worry about a lot.
And the idea in this problem is
that there's some discipline,
say computer science.
And if you look in all the
universities in the world,
there's M faculty lines.
So the total number of jobs is
M, and that's a fixed value.
Because of budgetary
constraints,
universities aren't
growing, and so they're not
going to create more computer
science faculty positions.
It's going to stay
fixed over time.
So it's a constant.
And every year, professors
generate more graduates
who become professors.
And in particular, we're going
to assume that in this field
every professor
graduates 1 student who
goes on to become a professor,
or tries to if there's jobs.
So each professor
generates 1 graduate
who becomes a new professor, as
long as there's jobs, per year.
With one exception, and
that's first-year professors
because they're too busy
learning how to teach,
getting grants, doing
administrative stuff,
just figuring out
how it all works.
So they don't have time to
produce any grad students.
So first year or new professors
don't produce anything.
Except first-year profs--
oops, let me correct that.
So first-year profs have 0.
Now, matters are made
worse by the fact
that Congress passed a
law, and this is true,
that more or less bars mandatory
retirements in colleges.
And so that means there are no
retirements in this problem,
and we're going to assume
the faculty live forever.
And so once you fill a
position, it's filled forever.
It doesn't exist anymore.
In fact, if you walk
around the math department,
you can see the impact of this.
[LAUGHTER]
I think the median age
is now well into the 70s.
And there will be actually a
phenomenon over the next 10
years as the math faculty
progress into their 80s
where they actually, in reality,
probably do start to retire.
And there will be
a whole new wave
of people hired in mathematics,
just as happened back
in the '50s and '60s after
the Sputnik crisis where
a lot of mathematicians
were hired.
And then they stayed in those
positions, for our purposes,
forever.
Now, the question is, when
do all the jobs get filled?
So when are all M jobs
filled by this process?
Now, to be able to
answer this question,
we need one more
piece of information.
Can anybody think
about one more fact
here that we need before we
start going off and answering
this question, analyzing it?
Yeah?
AUDIENCE: How many
professors do we start with?
PROFESSOR: How many
professors do we start with?
What's the boundary condition?
What's the base case if we
were doing an induction?
All right.
So let's say the
boundary condition,
and this always is
important with recurrences,
is that the first professor
is hired in year 1,
and there were none
before that person.
All right?
So in year 1
there's 1 professor,
and that professor's new.
OK.
So now we have all
the information
necessary to solve the problem.
So let's do that and
set up a recurrence.
We're going to define f of n
to be the number of professors
during year n.
And we know from our boundary
condition that in year 0
there were none, and in year
1 there was 1, a new one.
What is f of 2?
How many professors
are there in year 2?
AUDIENCE: 1.
PROFESSOR: 1, because
the one that was there
was too young to do anything.
So he or she is the only one
left, all right, in year 2.
What is f of 3?
How many profs are
there in year 3?
2-- the one you had, and by that
point that one is old enough
to produce a new one.
All right.
What's f of 4?
AUDIENCE: 3.
PROFESSOR: 3, the 2
you had, and there
was one who's been there 2
years to produce a new one.
All right.
And let's do one more, f of 5?
AUDIENCE: 5.
PROFESSOR: 5.
These guys produced 1 each,
and you had 3 existing.
All right.
So we can actually write
down the recurrence now
by sort of the process
we just went through.
For years 2 and beyond, the
number of professors in year n
is the number that we
had last year-- that's
the previous ones-- plus the
number that were generated,
new professors, the graduate
students that graduated.
In terms of f, how many
new ones are there?
AUDIENCE: f of n minus 2.
PROFESSOR: f of n
minus 2, because that's
the number of professors that
were there a couple years ago,
and they are now
generating them.
So that's the new ones there.
Do people recognize
that recurrence?
Yeah, it's pretty famous.
How many people have not seen?
This is called the
Fibonacci recurrence.
It produces the
Fibonacci numbers.
How many people
have not seen it?
Yeah, very famous.
Yeah, pretty much
everybody has seen that.
Actually, this is
the first recurrence
that was known to be
studied of all recurrences.
It was published by
Leonardo Fibonacci of Pisa
in 1202, all right,
so over 800 years ago,
and he studied it for
modeling the population
growth of rabbits.
And the idea is that you
have a pair of rabbits,
and in every month
after their first year
of life-- sorry,
first month of life--
they produce two new rabbits.
All right?
So it's the same notion as here.
The first pair you do
nothing, but after that you're
reproducing one for one.
And that's an
abstraction, but it
produces the same recurrence.
Now, Fibonacci is credited
with discovering it.
Really that means he's the
one that told the Europeans
about it back then.
And in fact, it's now been
traced back in to about 200 BC.
The Indian
mathematicians knew all
about Fibonacci's
recurrence, and they
were using it to study certain
properties of grammar and music
way back at 200 BC.
This recurrence comes up in
all sorts of applications.
Kepler used it in
the 16th century
while studying how
the leaves of a flower
are arranged around the
stem-- how many leaves
you have in sort of each level
coming out around a stem.
The first solution was
discovered by de Moivre
in the 18th century.
And we're going to talk about
how to solve this in a minute.
But he was the first one
to figure out a closed form
expression for f of n.
Lame used it in the
19th century when
he was studying the
Euclidean GCD algorithm.
You know that pulverizer
thing and doing GCDs?
It turns out that if you want
to analyze the running time,
well, you get a Fibonacci
recurrence comes
into play there, and that was
discovered in the 19th century.
In the 20th century, it was
used in the study of optics,
economics, and algorithms, and
it was named for Fibonacci.
It got a name in
the 19th century.
In fact, this is
so popular and used
in so many places there is a
journal in mathematics called
the Fibonacci Quarterly
Journal where they
study these kinds of things.
So today what we're going to
do is solve this recurrence
and actually solve a much
broader family of recurrences
called linear recurrences.
And we're going to
get a closed form.
I mean, you can produce
the Fibonacci numbers one
after another, but we're
going to derive a formula
for the n-th Fibonacci number.
And when we're going
to do it, we're
going to do it more broadly for
a class of linear recurrences.
So let me define what that is.
So a recurrence is
said to be linear
if it is of the form f of n
equals a constant a1 times
f of n minus 1 plus a2 times
f of n minus 2 plus dot dot
dot d'th constant ad
times f of n minus d.
And we could simplify
that as the sum
i equals 1 to d of a sub
i times f of n minus i.
And the constants
are fixed here,
so for fixed a sub i and d.
So the number of terms
has to be a constant,
and each coefficient
has to be a constant.
Can't vary.
And we define d to be the
order of the recurrence.
So d is the order
of the recurrence.
OK?
And you can see, of course,
that Fibonacci's recurrence
is linear.
What's its order?
2.
And the coefficients,
the a's, are just 1.
So it's a simple
linear recurrence.
All right.
So let's see how to solve it.
Well, actually,
before we do that,
can you see the difference
between this recurrence,
linear, and divide-and-conquer
recurrences?
Right?
What do I have here inside for
a divide-and-conquer recurrence?
I get a fraction of n, right?
And here, I'm
subtracting a constant
from n, usually like 1,
2, 3, an integer from n.
So linear is when inside you
have n minus 1, n minus 2.
Divide and conquer,
you got n/2 or 3/4 n.
And it makes a huge
difference in the solution.
All right.
So what I'm going to do is
give you a closed-form solution
for these kinds of recurrences.
And it won't be completely
easy because it took Europeans
six centuries to find
the solution to this.
Right?
Fibonacci discovers
the thing in 1200
and tells everybody about it.
And it wasn't until
600 years later
that they figured out
a closed-form solution.
So let's do that.
Now, what we're going to
do to do it this first time
ourselves, and we
don't have the formula,
is to use guess and verify.
So we're going to
guess a solution
and check that it works.
And we're going to guess,
really, a class of solutions.
We're going to try f of n is
an exponential in n, alpha
to the n for some
constant alpha.
Now, we're going
to figure out what
alpha is as we go along during
the verification process.
All right?
So let's try to verify
this guess and plug it in.
We know that f of n is f of n
minus 1 plus f of n minus 2.
Well, let's plug that
in and see what we get.
That gives us alpha to the
n equals alpha to the n
minus 1 plus alpha
to the n minus 2.
All right?
Now, that means I can divide
by alpha to the n minus 2,
and I get alpha squared
equals alpha plus 1.
And now I can use the quadratic
formula to solve for alpha.
All right?
That means that alpha
squared minus alpha minus 1
equals 0, which means that
alpha equals-- minus minus 1
is 1 plus or minus the square
root of-- minus 1 squared
is 1 minus 4ac plus 4 over 2.
OK?
So there's two possible
solutions here.
This is 1 plus or minus the
square root of 5 over 2.
So it works if f of n
equals either-- well, we'll
call the roots here
alpha 1 and alpha 2.
Alpha 1 will be the
positive case, 1
plus square root of 5 over 2.
And alpha 2 will be
the negative case, 1
minus square root of 5 over 2.
All right?
So guess and verify works so
far if we have an exponential
with either one of those bases.
All right?
By the way, does anybody
recognize that number?
It's a famous number.
AUDIENCE: Golden ratio.
PROFESSOR: The
golden ratio, which
is supposed to have all these
magical mystical properties.
That when you look
at a building,
if its aspect ratio is that,
it's perfect to the human eye.
I don't know.
But there's a lot of stuff
about the golden ratio that
happens to come up here.
OK?
Now in fact, there's more
than just these two solutions.
It turns out that whenever
you have a linear recurrence
and you've got two or
more solutions like that,
any linear combination
is also a solution.
All right?
So we're just going to
state that as a fact.
It's not too hard to prove,
but we won't prove it in class.
So if f of n equals
alpha 1 to the n--
and this is true for any alpha
1 and alpha 2-- and f of n
equals alpha 2 to
the n are solutions
to a linear recurrence--
and here I mean
without yet applying
the boundary conditions.
So far, we've ignored
the boundary conditions,
and we'll just do that
for a little longer.
So if there are solutions
without worrying
about the boundary
conditions, then f of n
equals c1 times alpha 1 to
the n plus c2 alpha 2 to the n
is also a solution for
any constants c1 and c2.
All right?
So any linear combination
of our solutions also works.
If you plugged it in to do
verify, it would be fine.
So that means that f
of n equals c1 times
1 plus square root
of 5 over 2 to the n
plus c2 1 minus square
root of 5 over 2 to the n
is a solution-- oops--
to Fibonacci's recurrence
without boundary
conditions again.
All right?
So I could plug this
expression into the recurrence,
and it would satisfy it.
I won't do that.
But we haven't dealt with
the boundary conditions yet.
And in fact, dealing with
the boundary conditions
is what determines the
values of these constants.
I could have Fibonacci's
recurrence where f of 0 was 10
and f of 1 was 20
if I wanted to,
and then the recurrence
would be the same afterwards.
And it will turn out I get
different constants here.
All right?
But otherwise, the form is
going to look like this.
So let's see how
to make that work.
Let's see how to determine
the constant factors.
OK.
So to determine the
constant factors,
we plug in the
boundary conditions.
So we have f of 0 equals 0
from the boundary condition.
And now we plug that into
our formula over there.
That's c1 times alpha 1 to the
0 plus c2 alpha 2 to the 0.
Of course, anything
to the 0 is just 1.
And that means that
c2 equals minus c1.
All right?
And now I'll use the
next boundary condition
to nail them down.
All right?
So I know also that
f(1) is 1, and that
equals c1 1 plus square root
of 5 over 2 to the first power
plus c2 1 minus the square root
of 5 over 2 to the first power.
I plug in c2 as minus c1.
So I get c1 1 plus square
root of 5 over 2 minus c1 1
minus square root of 5 over 2.
And now that I can
factor out the c1,
I get 1 minus 1 square root of
5 minus minus square root of 5,
which gives me c1 2
square root of 5 over 2.
These cancel.
This was all equal to 1, so
that means that c1 equals 1
over the square root of 5.
And of course, c2 is minus that.
c2 is minus 1 over
the square root of 5.
Any questions so far?
All right.
Now I could write
out the formula
for the Fibonacci numbers.
All right?
So the solution is f
of n equals c1, which
is 1 over square root of
5, times the n-th power
of the first root plus
c2, which is minus 1
over square root of 5, times the
n-th power of the second root.
And that is the formula for
the n-th Fibonacci number.
You wouldn't have guessed
that to start with obviously.
That would require pretty
divine inspiration.
And you can sort of see why it
took them 600 years in Europe
to figure out the answer.
All right?
It's not the first
thing you'd think about.
In fact, if somebody told you
the answer and said this is it,
you'd go, oh, give me a break.
All right?
It does not look like-- I
mean, what are the chances that
evaluates to an integer?
All right?
It's got square root of
5's all over the place.
Right?
And here I'm telling
you that f of 6,
the sixth Fibonacci number,
which is 8, 3 plus 5,
I'm telling you that is equal
to 1 over square root of 5 1
plus the square root of 5 over
2 to the sixth power minus 1
over square root of 5 1 minus
the square root of 5 over 2
to the sixth power.
I mean, would you believe
me if I told you that?
Probably not.
What are the
chances that's true?
Somehow, magically, all those
square root of 5's all go away,
and you're just left
with 8, a simple integer.
OK?
All right.
Yeah, there's sort of some more
interesting things about this.
What happens to this value
here as n gets large?
What does it do as n gets large?
AUDIENCE: Gets small.
PROFESSOR: It gets small Because
1 minus the square root of 5
over 2 is about 0.6.
This is about, here, this is
about 0.618 something or other.
And if I take a fraction less
than 1 to the n-th power,
it goes to 0.
It goes away.
So in fact, as n
gets large, this
is what the n-th Fibonacci
number starts to look like.
In particular, f of n equals
just that first one-- 1
over square root of 5 times the
golden ratio to the n-th power
plus some error term, delta n.
And delta n is less than a
1/10 for n greater than 4.
And it is little l of 1
in general going to 0.
So in fact, the n-th
Fibonacci number
is about the n-th power of
the golden ratio divided
by square root of 5.
Yeah?
AUDIENCE: [INAUDIBLE]
top of the right board--
PROFESSOR: Yeah?
AUDIENCE: --you say that you can
pick sort of any constants that
satisfy it?
PROFESSOR: Yep.
AUDIENCE: Does that
shift the sequence
over, or what does that do?
I guess I don't understand.
PROFESSOR: OK.
You're asking, what are
these things doing here?
AUDIENCE: Yeah.
And why [? isn't there-- ?]
why are they also solutions?
Because this is the right ones?
PROFESSOR: The right one.
OK.
So there's two parts
to a recurrence.
There's this part,
OK, and there's
the boundary conditions-- f(0)
equals 0 and f(1) equals 1.
There are really
two parts to it.
If you change the
boundary conditions,
you would change the
rest of the terms, right?
If I started with
10 and 20, f of 2
would be 30, OK, if I'm using
this recurrence form for n
bigger than or equal to 2.
Now, so when I'm computing a
solution, first I'm saying,
let's ignore the
boundary conditions
and look at all the recurrences
that share this part.
And that family of
recurrences, I'm claiming,
have these solutions.
This is the solution
space if I just
worry about this part and
not the boundary conditions.
Now, once I plug the
boundary conditions in,
that determines which
values of c1 and c2 I get.
And so if I take the version
of the recurrence where f(0)
is 0 and f(1) is 1, then c1
is going to be 1 over root 5,
and c2 is going to be
negative 1 over root 5.
If I use this
recurrence with f(0)
equals 10 and f(1) equals 20,
I get different constants,
but these powers stay the same.
In fact, maybe we'll make
an exercise like that
from the problem set
to figure out what it
is if I started with 10 and 20.
OK?
So that's what
the constants have
to do-- they come from the
boundary condition, which
is sort of a cool fact.
Does that make sense?
AUDIENCE: Yeah.
PROFESSOR: Any other questions
about what I was doing?
So we used guess and verify, but
I sort of guessed a form first.
And as I verified, I sort of
revised my guess along the way.
All right?
So I started guessing this.
And then I said,
well, let's refine
that guess so that really
it's one of these two guys.
And then I said-- I
used a fact that I
didn't prove-- that
said I could use really
any linear combination,
plugged it into the base cases,
and got my constants.
All right?
Now, this will lead up to
a formula or an approach
so you don't have to go
through this on your own.
You just plug in the
formula in general.
OK.
Let's get back to the
original question of when
all the jobs are filled.
For what value of n is
f of n bigger than M?
All right?
Let's do that.
OK.
To see when all the jobs
are filled, all M jobs,
for the n when f of n is
bigger and equal to M,
when we got a Fibonacci
number that's bigger than M.
And we just showed
that basically f of n,
well, it equals this
where that's a tiny thing.
All right?
So we need to figure out when
is 1 over square root of 5 times
the golden ratio to
the n-th power-- plus
I have this tiny little
thing that doesn't matter--
is bigger than or equal to M?
So I can solve for n now by
subtracting the delta term,
multiplying by square root of 5.
That gives me the golden
ratio to the n-th power
bigger than or equal
to square root 5 times
M minus the tiny thing.
Now I take logs.
And I'm in trouble when n is
bigger than or equal to log
of this over log of the base.
In other words, this is theta.
What's this theta of here?
That goes away.
I can skip all the
constant factors.
What's this theta of
here, this expression,
in terms of theta of something
to do with some function of M?
AUDIENCE: Log of M?
PROFESSOR: Log of M. So the jobs
get filled in log of M years.
And in fact, if I
plugged in, for example,
M equals 10,000 into
this expression,
I would find that
all the jobs are
filled in 20 years, which
is more or less what
happened in computer science.
Ballpark, those numbers
are roughly correct.
All right?
But you can solve it exactly now
by just plugging in whatever M
you want.
That's a fraction
less than 1/10.
And you can solve
to find out when
your population gets that big.
OK.
Any questions?
OK.
So now what I want
to do is show you
how to use this same idea to
solve any linear recurrence,
and this is the process
you'll go through to do it.
All right.
So we're going to solve a
general linear recurrence.
So in this case, we've got
f of n is the sum from i
equals 1 to d, for an
order d recurrence,
a sub i f of n minus i.
And we've got to have
boundary conditions.
So we'll have f of 0
is b0, f of 1 is b1.
And how many boundary
conditions do
you think I'm going
to need to have?
Any guesses?
I had two for Fibonacci.
In this case, I've got d terms.
AUDIENCE: d.
PROFESSOR: d.
I'm going to need d
boundary conditions.
So we'll go all the
way to f of d minus 1
equal to b of d minus 1.
All right?
And now I'm going to try
f of n is alpha to the n.
We're going to plug it
into this expression.
And then when we do,
we get alpha to the n
equals a1 alpha to
the n minus 1 plus
a2 alpha to the n minus 2
all the way down to a sub
d alpha to the n minus d.
All right?
Just plugging into there.
I can divide everything
by alpha to the n minus d,
and that gives me
alpha to the d equals
a1 alpha to the d minus 1 a2
alpha to the d minus 2 ad times
alpha to the 0, which is just 1.
And I can rewrite this as
a polynomial equal to 0.
So that means that
alpha to the d minus
a1 alpha to the d minus 1
minus a2 alpha to the d minus 2
minus ad is 0.
This is called the
characteristic equation
of the recurrence.
OK?
Now, what you need to do is
compute the roots of this,
and we'll use the roots to get
the solution to the recurrence.
Now, the simple case
is when all the roots
are different-- so let's
do that first-- like there
was in the Fibonacci example.
All d roots are
different, and let's call
them alpha 1 to alpha d.
In that case, the solution
is all linear combinations
of their n-th
powers, just like it
was with Fibonacci without
the boundary conditions.
c1 alpha 1 to the
n plus c2 alpha 2
to the n plus cd
alpha d to the n.
All right?
Same thing happens that
happened in Fibonacci,
that this becomes the
solution before the boundary
conditions are applied.
All right?
And by the way, if your
characteristic equation has
imaginary roots, that's fine.
It doesn't matter
because the imaginary i
term will disappear just
like the square root of 5
disappeared.
Has to because we know f of n,
in this case, is a real number.
All right?
OK.
Now, to find out the values
of the coefficients, what
are we going to do?
What is it?
AUDIENCE: Boundary conditions.
PROFESSOR: Boundary
conditions, yep.
So now we solve for c1, c2, cd
from the boundary conditions.
That is, f of i is b
of i for i from 0 to d.
So for example, we
know that f of 0
is c1 times alpha 1 to the
0, which is 1, c2 alpha 2
to the 0 plus cd.
And that's going to be
equal to b0 and so forth.
So you get a system of
equations, d equations
in d variables, that you solve
to find the coefficients.
And that will give the
unique and correct solution
to the recurrence.
All right?
Now, turns out this system of
equations is never degenerate.
It always has a solution.
And you can prove that,
but we won't do that.
Any questions about
how to do that?
All right.
Yeah?
AUDIENCE: Sorry, I just
had a quick question.
But what exactly was the
distance there between n and d?
d is like the degree of--
which the terms you go back to.
PROFESSOR: Yes, Yeah. d is
how far back you're going.
You go as far back
as f of n minus d.
AUDIENCE: OK.
Cool.
PROFESSOR: And that
becomes the degree,
and that is a
constant-- 2, 3, 4.
Well, I don't think we'll
ever ask you anything beyond 4
because it gets to
be a pain to do.
Typically, it's 2 or 3,
sometimes even just 1.
And then that
becomes the power of
your characteristic equation.
It's the order of the
characteristic equation.
Any other questions?
All right.
That was the nice case
where all the roots
of your characteristic
equation were different.
If they're not all different,
it's a little more complicated.
So the tricky case
is repeated roots.
Now, the theorem, which we won't
prove but tells you what to do,
is that if alpha is a root of
the characteristic equation
and it is repeated r times-- so
x minus alpha to the r-th power
is a factor-- then alpha to
the n, n times alpha to the n,
n squared times alpha
to the n, all the way up
to n to the r minus 1
times alpha to the n
are all solutions
to the recurrence.
All right?
And then you would
treat them just
as you would the other roots.
You take linear
combinations, just
like you did with the other
roots to put it all together.
By the way, is anybody starting
to recognize a similarity
with something else that
you've studied in the past?
AUDIENCE: Differential
equations.
PROFESSOR:
Differential equations.
This is the discrete analog
of differential equations.
All right?
Recurrences is the same thing.
All the math we're
going to do henceforth
is going to look just
like what you did
with differential equations.
So that's sort of good news and,
I guess, bad news, depending
on whether you like that stuff.
AUDIENCE: Bad news.
PROFESSOR: Bad news.
Yeah, OK.
All right, let's do an
example that uses maybe
the repeated roots case.
Suppose there's a
plant out there,
and this plant lives
forever but only reproduces
in the first year of life
and then never again.
And it reproduces one for one.
All right?
Let's see what
happens in that case.
Actually, there's a plant
sort of like this in Hawaii.
I think it's called the
century plant, and it's rare.
We'll see why when we
solve this problem.
So it reproduces one for one
during the first year of life,
then never again, and
the plant lives forever.
So our question is, how fast
does the plant population grow?
All right?
In year n, how many
plants there are.
So to figure that out, we're
going to set up a recurrence.
We're going to let f of n be
the number of plants in year n.
And we're going to say that the
first plant miraculously comes
into existence in year 1.
So that in year 0 there's
none of them, and in year 1
there's 1.
And now we want to know how
many there are in year n.
OK.
So let's set up the recurrence.
f of n equals-- well,
the previous year,
how many plants were
there in terms of f?
AUDIENCE: [INAUDIBLE]
PROFESSOR: f of n minus 1.
All right?
That's how many
there were last year,
and they're all
still alive, plus
we've got to add the
number of new plants.
Well, that would
be-- yeah, well,
how many new plants are there?
It's all the plants that
were one-year-old last year.
So that's all the
ones that were alive
last year minus the ones that
were alive the year before.
That's how many new plants
there were last year,
and they each produce
1 new one this year.
Is that OK?
Everybody buy that?
OK.
So that equals 2 f of n
minus 1 minus f of n minus 2.
OK.
What's my characteristic
equation for this recurrence?
Yeah?
AUDIENCE: Alpha squared
minus 2 alpha plus 1.
PROFESSOR: Yes.
OK?
alpha squared minus
2 alpha plus 1.
All right?
We jumped to that answer.
I could have written down alpha
to the n minus 2 times alpha
to the n minus 1 plus
alpha to the n minus 2
and then divided by
alpha to the n minus 2.
But pretty quickly, you want
to start just reading it off.
That's the
characteristic equation.
I get alpha squared, and
then bringing this over,
a minus 2 alpha, bring
this over, plus 1 equals 0.
Any questions on that?
This is the process
now you will always
use that we're going through.
All right.
What are the roots
to this thing?
What are the roots to the
characteristic equation?
AUDIENCE: Alpha
minus 1 [INAUDIBLE].
PROFESSOR: Yeah.
Alpha minus 1
times alpha minus 1
is 0, which means alpha equals?
AUDIENCE: 1.
PROFESSOR: 1.
So the roots are alpha equals
1, and it's a double root.
OK?
It's a double root.
It occurs twice because this
is alpha minus 1 squared.
OK.
So what solutions am I going
to use to my recurrence?
What's f of n going
to be now before I
put in the boundary conditions?
Well, it's going to be
c1 times what to the n?
AUDIENCE: 1.
PROFESSOR: 1.
That's the first root.
What's here?
What's the next thing
I'm going to put here?
AUDIENCE: [INAUDIBLE].
PROFESSOR: n times 1 to the n.
All right?
Because I've got a root
that's repeated twice,
r equals 2, so I have alpha
to the n and n alpha to the n.
Just happens alpha's 1,
which makes it really easy.
All right?
So this is now my solution
before the boundary conditions.
In fact, that gets even simpler.
That's 1.
That's c1 plus c2 times n.
And now all that's
left is to plug
in the boundary conditions.
So let's do that.
Well, f of 0 equals 0.
If I plug it into here,
what happens for f of 0?
What does that become?
AUDIENCE: [INAUDIBLE].
PROFESSOR: C1.
So c1 equals 0.
That's good.
f of 1 equals 1.
And now that's going
to be c1 plus c2.
c1 is 0, so it
means c2 equals 1.
All right.
So this is really easy.
What's f of n equal?
n.
So we went through a lot of work
to get an answer that probably
we could have guessed
pretty easily by just
plugging in a few examples.
But the nice thing is this
works for all the cases
when it's not so easy to guess
by plugging in the examples.
Yeah, now you can see why this
is a relatively rare plant.
Its population is
growing very slowly.
OK.
Any questions about that?
That had repeated roots.
All right.
Yeah?
AUDIENCE: So we just
guessed that f of n
is equal to alpha
to the n, and we
don't know if that works for
like different [INAUDIBLE].
But how do you know that it
works until you plug it in?
If you tried it out [INAUDIBLE]?
PROFESSOR: Yeah.
We didn't go through
and prove it.
We used some facts along the
way that we didn't prove,
which you can do.
Like, in particular, the fact we
didn't prove that theorem there
that those will be the roots.
And we didn't prove that
any linear combination
would be a solution.
But those are facts you could
take from-- you could actually
prove them.
They're not too hard to prove.
But otherwise, I think we
went through every step
and narrowed it in.
If we wanted to be
really sure, we'd
go back and prove
it by induction
against the original recurrence.
And we'd say this would be
the induction hypothesis.
We'd cover the base
cases-- f of 0 and f of 1.
And then we'd plug this into
the recurrence up there,
which is, does n equal twice
n minus 1 minus n minus 2,
is what we would do.
And you see that it's true.
So you could really
prove it by induction
for any of the
solutions we ever get.
And if you worked this
way to get the solution,
it will always work.
This method never fails,
gives you the wrong answer.
In fact, that's an
important point.
When we tell you to solve a
recurrence using this method
on an exam or for
homework, you don't
have to verify it by
induction unless we
say verify by induction.
Because we're
giving you a method
that is guaranteed to work.
And if you do the method
right, you're fine.
There's no guessing here so far.
In a minute, we're going
to do a little guessing,
but not so far.
Any other questions?
OK.
This works for all homogeneous
linear recurrences.
All right?
Remember homogeneous
and inhomogeneous
from differential equations?
It's the same
thing happens here.
And now we're going to talk
about the inhomogeneous case.
All right.
So we've been looking at
linear recurrences, which
means you have something like
f of n minus a1 f of n minus 1
minus ad f of n
minus d equals 0,
and that means it's homogeneous.
Now, instead of 0, I might
have had something else
here, might have
been equal to 1,
maybe n squared, or some
general function g of n.
These cases are
all inhomogeneous.
OK?
Now, solving inhomogeneous
linear equations
is just one step harder
than homogeneous.
Yeah?
AUDIENCE: All right.
Could you define homogeneous
and inhomogeneous
for people who didn't take
differential equations?
PROFESSOR: Yeah.
Homogeneous means
you have the 0 here.
You got all these f
terms here, right,
and there's nothing else.
f of n minus a1 f n
minus 1 dot dot dot
minus ad f n minus d equals 0.
Like with Fibonacci
numbers-- f of n minus f n
minus 1 minus f n minus 2 is 0.
OK?
I could consider
other recurrences
where there's something else
out here that it equals.
Like I could have a
Fibonacci recurrence
where I have f(n) equals f of
n minus 1 plus f(n) minus 2
plus g of n, like n cubed.
That would give me a
Fibonacci-like recurrence.
And as soon as you put
a non-zero out here,
then it's inhomogeneous.
And now I'm going
to tell you how
to solve that kind
of recurrence,
which is more general.
OK?
OK.
So let me outline the method,
and then we'll do an example.
All right.
So the general inhomogeneous
recurrence is exactly this.
It's f of n minus a1 f of n
minus 1 minus a sub d f of n
minus d equals g of n, where
that's some fixed function
of n, nothing to do with f.
And we solve it according to
the following three-step method.
In step 1, we
replace g of n by 0,
thereby creating the situation
we already know how to solve.
And we solve the
homogeneous recurrence--
because you got a 0 there now--
but we don't go all the way.
We ignore the boundary
conditions for now.
So you don't get all the
way to the final answer
where you plugged in
the boundary conditions
to get the constant
coefficients.
Leave the constant
coefficients undecided.
Then in step 2, we
put back in g of n,
and we find any what's called
particular solution, again
ignoring boundary conditions.
So you'll have the
constant factors there too.
All right?
And then step 3 is going to be
to put the whole thing together
and plug in the boundary
conditions and solve it.
So we add the homogeneous
and particular--
and I'll show you how to
do step 2 in a minute--
so we add the homogeneous and
particular solutions together
and then use the boundary
conditions to get the answer.
OK.
So let's do an example.
Let's go down here and do an
example of the three steps.
So let's say that
our recurrence is
this nasty-looking thing-- f
of n equals 4 f of n minus 1
plus 3 to the n.
And the boundary condition
is that f of 1 is 1.
So step 1 says, ignore
this 3 to the n thing
and get back to just the
homogeneous form, which
is f of n minus 4 f
of n minus 1 is 0.
So let's solve that.
What's the
characteristic equation
for the homogeneous part?
What's the
characteristic equation?
What is it?
AUDIENCE: Alpha [INAUDIBLE].
PROFESSOR: Close.
It's-- you could either say
alpha to the n minus 4 alpha
to the n minus 1.
But better to simplify it
into an order 1 polynomial.
So it would be alpha minus 4,
really simple in this case.
So the characteristic
equation is alpha minus 4,
alpha minus 4, all
right, equals 0.
And it's really easy to
find the root of this thing.
It's just alpha equals
4 is your only root.
And that means the
homogeneous solution
is f of n equals a
constant times 4 to the n.
All right?
So that's step 1.
OK.
Let's do step 2, which
we've not tried before.
We need to find a
particular solution,
and this just means
any old solution
to f of n minus 4 f of n
minus 1 equals 3 to the n,
without worrying about
boundary conditions.
Now, there's basic rules
to use to figure out
what to guess here.
And basically, the idea is that
you guess something for f of n
that looks a whole lot
like this g term out here.
And in particular, if
this g term is 3 to the n,
you guess a constant
times 3 to the n.
If it's 5 to the n, you guess
a constant times 5 to the n.
If it's n squared, you guess a
polynomial that's of degree 2
and so forth.
OK?
So let's guess a
constant times 3 to the n
and see if we can
make it work, not
worrying about the
boundary conditions.
All right.
And this is just like
differential equations, right?
Same guessing
strategy exactly you
use in differential equations,
those of you who've had that.
So we guess f of n equals a
constant times 3 to the n,
and let's plug it in.
So we plug that in up there.
We get c3 to the n minus
c3 to the n minus 1
equals 3 to the n.
All right?
So let's divide by
3 to the n here.
We get 3c minus c equals 3.
Did I do that right?
No, I left off my 4 here,
right, left off that.
So there's a 4 here.
And that means
that c is minus 3.
OK?
So I got minus c equals
3, so c is minus 3.
All right.
That means that the particular
solution is just f of n
equals-- c is minus 3--
minus 3 to the n plus 1.
All right?
So now I found a solution where
there's no constants involved.
This time, the
constant went away just
plugging into the
recurrence formula.
I didn't use base cases yet.
I just found a
particular case that
works for the recurrence--
minus 3 to the n plus 1.
OK?
All right.
Now we go to step 3.
All right.
Step 3, we find the
general solution
by adding the homogeneous
and the particular solution.
So we take f of
n equals c1 times
4 to the n plus negative
3 to the n plus 1.
And all I've got to
do is determine c1,
and I do that from the
boundary conditions.
Yeah, let's see.
Where was my boundary
condition here?
Ah, up there-- f of 1 is 1.
So f of 1 equals 1.
Plugging in 1 here,
I get c1 times 4
to the 1 minus 3 squared.
All right?
So I have 4c1 minus 9.
Put the 9 over here,
I get 10 equals 4c1.
And that means that c1 is 5/2.
And now I know the
final solution for f(n).
It's 5/2 4 to the n
minus 3 to the n plus 1.
Now, when you're
all done with this,
you don't have--
we won't make you
do an inductive proof, which you
could do to verify it's right.
If you wanted to be
really, really careful,
you should do that.
But it is a good idea just
to check a couple values of n
to make sure you didn't make a
mistake because you might have.
With all these calculations,
you might have made a mistake.
So let's check, for
example, f of 2.
So by the recurrence,
f of 2 is 4 times
f of 1-- it's 4 times 1-- plus
3 squared, which is 9, is 13.
And let's just check
when we plug in 2 here.
f of 2 is 5/2 times 16 minus 27.
5 times 8 is 40, minus 27 is 13.
Just as a sanity check.
Because there's a decent
chance if you made a mistake,
it'll catch it pretty
quick with n equals 2 or 3.
All right?
Just to make sure
you got it right.
Any questions about that?
So the tricky part
here is guessing
the particular solution.
So let me give you the rules
for that, just write those down,
and maybe I'll do
one last example.
All right.
So we're guessing a
particular solution.
So if g-- that's saying
the non-homogeneous part--
is exponential, you should
try guessing an exponential
of the same type.
So for example, say that g is
2 to the n plus 3 to the n.
What you should do is guess
some constant a times 2 to the n
plus some constant
b times 3 to the n.
Plug it in-- not to the
boundary conditions.
Plug it into the recurrence
equation and solve for a and b.
And it generally will work.
If g is polynomial, you
should guess a polynomial
of the same degree.
So for example, say g of
n is n squared minus 1.
You should guess-- let's
see-- guess an squared
plus bn plus c.
Plug that in for f of n.
All right?
So when we're doing these
guesses, we're guessing f of n.
Same thing up here.
Now, say you mixed two together.
Suppose you have an
example where g of n
equals 2 to the n plus n.
What do you suppose
you do in that case?
AUDIENCE: [INAUDIBLE]
to guess [INAUDIBLE].
PROFESSOR: Yeah, guess each one
separately, add them together.
So you're going to guess
f of n equals a times 2
to the n plus bn plus c.
All right?
Because you take the guess for
that plus the guess for that.
All right.
Now, there's one last thing
that can go wrong here.
And that is you can try your
guess, and it doesn't work.
All right?
So there's rules for that too.
For example, if, say,
g(n) is 2 to the n
and your guess of a times 2 to
the n, where a is a constant,
fails.
And we'll do an example in
a minute where it fails.
What you do then is you guess
a polynomial times 2 to the n.
So you'd guess an plus
b times 2 to the n.
And if that fails,
your next guess
would be an squared plus
bn plus c times 2 to the n.
And it won't happen,
but if that failed,
you'd guess a cubic
times 2 to the n.
You keep pounding it with
another factor of n in front
if the guess fails.
And that's true
for anything at all
you would be doing like
this, and that'll work.
And I don't think I've ever
encountered an example where
you have to go very
far to make that fly.
Same thing we had
for repeated roots
in the characteristic equation.
Multiply those factors
of n in front of it
to get to the answer.
All right.
Let's do one more
example where it fails,
so we get to see what happens.
Let's try this recurrence--
f of n is 2 f of n minus 1
plus 2 to the n.
And the boundary
condition is f of 0 is 1.
What's the first
thing I do for this?
AUDIENCE: Set it equal to 0.
PROFESSOR: Set it equal
to 0 and solve it.
Get the homogeneous solution.
OK.
So what's the
characteristic polynomial?
Alpha minus 2 is 0.
What's my root?
2.
And therefore, my
homogeneous solution
is c1 times 2 to the n.
Well, that's pretty simple.
What's the next step?
What do I have to find next?
AUDIENCE: Particular.
PROFESSOR: Particular solution.
All right.
So what am I going to guess?
a times 2 to the n.
All right?
So let's plug that in,
a times 2 to the n.
I'm going to plug it in to here.
So I get a 2 to the n
equals 2 times a times 2
to the n minus 1
plus 2 to the n.
Did I do that right?
Think so.
So I get 2 to the n, 2
to the n, 2 to the n.
I get a equals a plus 1.
Not so good.
All right?
There's no solution for a.
Well, that's bad.
All right.
So what do I do?
Any thoughts about what
I'm going to do next?
AUDIENCE: [INAUDIBLE].
PROFESSOR: What is it?
AUDIENCE: Change your guess.
PROFESSOR: Change the guess.
What's the next
guess going to be?
Yeah, all right.
So now I'm going to guess f
of n is an plus b 2 to the n.
We'll hope for better luck.
So let's plug that in.
an plus b 2 to the n into--
where am I, up there--
equals 2 a n minus 1 plus b
times 2 to the n minus 1 plus 2
to the n.
All right.
Now, I can divide out the 2 to
the n's here like I did before.
And I get an plus b equals
an minus a plus b plus 1.
That cancels here, b cancels.
So I got a solution--
a equals 1.
And I don't care what b is.
I didn't need to
set it to anything.
So I'll make it 0 just
to get it out of the way
because it didn't matter.
So my particular solution
then is f of n-- a is 1.
b I can just make 0 because
it didn't matter what I used,
so I'll make it simpler.
So I get n 2 to the n.
That's my particular solution.
And I've got my general
solution up there as c1 times 2
to the n-- sorry, the
homogeneous solution.
What's the next step?
What's step 3?
What do I do with these
guys, this solution
and that solution?
[? AUDIENCE: Add them. ?]
PROFESSOR: Add them together.
Good.
So the general solution is I
have f of n is the sum of c1 2
to the n plus the particular
solutions n 2 to the n.
How do I figure out what c1 is?
AUDIENCE: Plugging
in to [INAUDIBLE].
PROFESSOR: Plugging in the
boundary condition-- f(0)
equals 1.
f(0) equals 1.
Plug in 0, I get c1
2 to the 0 plus 0.
Well, that's pretty
easy. c1 equals 1.
So I now have the final answer.
f of n equals 2 to the
n plus n 2 to the n.
All right?
Any questions?
OK.
So it's a little
tedious to do this.
But the really nice thing
is any linear recurrence
you ever see, this method
always works, which is handy.
OK.
That's it for today.
