[MUSIC PLAYING BY J.S. BACH]
PROFESSOR: Hi.
You've seen that the job of
a programmer is to design
processes that accomplish
particular goals, such as
finding the square roots of
numbers or other sorts of
things you might want to do.
We haven't introduced
anything else yet.
Of course, the way in which a
programmer does this is by
constructing spells, which are
constructed out of procedures
and expressions.
And that these spells are
somehow direct a process to
accomplish the goal that was
intended by the programmer.
In order for the programmer to
do this effectively, he has to
understand the relationship
between the particular things
that he writes, these particular
spells, and the
behavior of the process that
he's attempting to control.
So what we're doing this
lecture is attempt to
establish that connection in
as clear a way as possible.
What we will particularly do is
understand how particular
patterns of procedures and
expressions cause particular
patterns of execution,
particular
behaviors from the processes.
Let's get down to that.
I'm going to start with
a very simple program.
This is a program to compute
the sum of the
squares of two numbers.
And we'll define the sum of the
squares of x and y to be
the sum of the square of x--
I'm going to write
it that way--
and the square of y where
the square of x is the
product of x and x.
Now, supposing I were to say
something to this, like, to
the system after having defined
these things, of the
form, the sum of the squares of
three and four, I am hoping
that I will get out a 25.
Because the square of three is
nine, and the square of four
is 16, and 25 is the
sum of those.
But how does that happen?
If we're going to understand
processes and how we control
them, then we have to have a
mapping from the mechanisms of
this procedure into the way in
which these processes behave.
What we're going to have is
a formal, or semi-formal,
mechanical model whereby you
understand how a machine
could, in fact, in principle,
do this.
Whether or not the actual
machine really does what I'm
about to tell you is completely
irrelevant at this moment.
In fact, this is an engineering
model in the same
way that, electrical resistor,
we write down a model v equals
i r, it's approximately true.
It's not really true.
If I put up current through
the resistor it goes boom.
So the voltage is not always
proportional to the current,
but for some purposes the
model is appropriate.
In particular, the model we're
going to describe right now,
which I call the substitution
model, is the simplest model
that we have for understanding
how procedures work and how
processes work.
How procedures yield
processes.
And that substitution model will
be accurate for most of
the things we'll be dealing
with in the next few days.
But eventually, it will become
impossible to sustain the
illusion that that's the way the
machine works, and we'll
go to other more specific and
particular models that will
show more detail.
OK, well, the first thing, of
course, is we say, what are
the things we have here?
We have some cryptic symbols.
And these cryptic symbols
are made out of pieces.
There are kinds of
expressions.
So let's write down
here the kinds of
expressions there are.
And we have-- and so far I
see things like numbers.
I see things like symbols
like that.
We have seen things before like
lambda expressions, but
they're not here.
I'm going to leave them out.
Lambda expressions, we'll
worry about them later.
Things like definitions.
Things like conditionals.
And finally, things
like combinations.
These kinds of expressions
are--
I'll worry about later--
these are special forms.
There are particular rules
for each of these.
I'm going to tell you, however,
the rules for doing a
general case.
How does one evaluate
a combination?
Because, in fact, over here,
all I really have are
combinations and some
symbols and numbers.
And the simple things like
a number, well, it
will evaluate to itself.
In the model I will
have for you, the
symbols will disappear.
They won't be there at the time
when you need them, when
you need to get at them.
So the only thing I really have
to explain to you is, how
do we evaluate combinations?
OK, let's see.
So first I want to get
the first slide.
Here is the rule for evaluating
an application.
What we have is a rule that
says, to evaluate a
combination, there are
two parts, three
parts to the rule.
The combination has
several parts.
It has operators and
it has operands.
The operator returns
into a procedure.
If we evaluate the operator,
we will get a procedure.
And you saw, for example, how
I'll type at the machine and
out came compound procedure
something or other.
And the operands produce
arguments.
Once we've gotten the operator
evaluated to get a procedure,
and the argument is evaluated
to get argument--
the operand's value
to get arguments--
we apply the procedure to these
arguments by copying the
body of the procedure, which
is the expression that the
procedure is defined
in terms of.
What is it supposed to do?
Substituting the argument
supplied for the formal
parameters of the procedure,
the formal parameters being
the names defined by the
declaration of the procedure.
Then we evaluate the resulting
new body, the body resulting
from copying the old body with
the substitutions made.
It's a very simple rule, and
we're going to do it very
formally for a little while.
Because for the next few
lectures, what I want you to
do is to say, if I don't
understand something, if I
don't understand something, be
very mechanical and do this.
So let's see.
Let's consider a particular
evaluation, the one we were
talking about before.
The sum of the squares
of three and three.
What does that mean?
It says, take--
well, I could find out what's
on the square-- it's some
procedure, and I'm not going
to worry about the
representation, and I'm not
going to write it on the
blackboard for you.
And I have that three represents
some number, but if
I have to repeat that number,
I can't tell you the number.
The number itself is some
abstract thing.
There's a numeral which
represents it, which I'll call
three, and I'll use that
in my substitution.
And four is also a number.
I'm going to substitute three
for x and four for y in the
body of this procedure that
you see over here.
Here's the body of
the procedure.
It corresponds to this
combination, which is an addition.
So what that reduces to, as a
reduction step, we call it, is
the sum of the square of three
and the square of four.
Now, what's the next step
I have to do here?
I say, well, I have
to evaluate this.
According to my rule, which you
just saw on that overhead
or slide, what we had
was that we have to
evaluate the operands--
and here are the operands,
here's one and
here's the next operand--
and how we have to evaluate
procedure.
The order doesn't matter.
And then we're going to apply
the procedure, which is plus,
and magically somehow that's
going to produce the answer.
I'm not to open up plus
and look inside of it.
However, in order to evaluate
the operand, let's pick some
arbitrary order and do them.
I'm going to go from
right to left.
Well, in order to evaluate
this operand, I have to
evaluate the parts of
it by the same rule.
And the parts are I have to find
out what square is-- it's
some procedure, which has
a formal parameter x.
And also, I have an operand
which is four, which I have to
substitute for x in the
body of square.
So the next step is basically to
say that this is the sum of
the square of three and the
product of four and four.
Of course, I could open up
asterisk if I liked--
the multiplication operation--
but I'm not going to do that.
I'm going to consider
that primitive.
And, of course, at any level of
detail, if you look inside
this machine, you're going to
find that there's multiple
levels below that that
you don't know about.
But one of the things we
have to learn how to
do is ignore details.
The key to understanding
complicated things is to know
what not to look at and
what not compute
and what not to think.
So we're going to stop this
one here and say, oh, yes,
this is the product
of two things.
We're going to do it now.
So this is nothing more than
the sum of the square
of three and 16.
And now I have another thing I
have to evaluate, but that
square of three, well,
it's the same thing.
That's the sum of the product
of three and three and 16,
which is the sum of nine
and 16, which is 25.
So now you see the basic method
of doing substitutions.
And I warn you that this is not
a perfect description of
what the computer does.
But it's a good enough
description for the problems
that we're going to have in the
next few lectures that you
should think about
this religiously.
And this is how the machine
works for now.
Later we'll get more detailed.
Now, of course, I made a
specific choice of the order
of evaluation here.
There are other possibilities.
If we go back to the telestrator
here and look at
the substitution rule, we see
that I evaluated the operator
to get the procedures, and I
evaluated the operands to get
the arguments first, before
I do the application.
It's entirely possible, and
there are alternate rules
called normal order evaluation
whereby you can do the
substitution of the expressions
which are the
operands for the formal
parameters
inside the body first.
And you'll get also
the same answer.
But right now, for concreteness,
and because this
is the way our machine really
does it, I'm going to give you
this rule, which has
a particular order.
But that order is to some
extent arbitrary, too.
In the long run, there are some
reasons why you might
pick one order or another,
and we'll get to that
later in the subject.
OK, well now the only other
thing I have to tell you about
just to understand what's going
on is let's look at the
rule for conditionals.
Conditionals are very simple,
and I'd like to examine this.
A conditional is something
that is if-- there's also
cond, of course--
but I'm going to give names to
the parts of the expression.
There's a predicate, which
is a thing that is
either true or false.
And there's a consequent, which
is the thing you do if
the predicate is true.
And there's an alternative,
which is the thing you do if
the predicate is false.
It's important, by the way, to
get names for, to get names
for, the parts of things, or
the parts of expressions.
One of the things that every
sorcerer will tell you is if
you have the name of a spirit,
you have power over it.
So you have to learn these names
so that we can discuss
these things.
So here we have a predicate,
a consequent, and an
alternative.
And, using such words, we see
that an if expression, the
problems you evaluate to the
predicate expression, if that
yields true, then you then go on
to evaluate the consequent.
Otherwise, you evaluate the
alternative expression.
So I'd like to illustrate that
now in the context of a
particular little program.
Going to write down a
program which we're
going to see many times.
This is the sum of x and y done
by what's called Peano
arithmetic, which is all we're
doing is incrementing and
decrementing.
And we're going to see this
for a little bit.
It's a very important program.
If x equals zero, then
the result is y.
Otherwise, this is the sum of
the decrement of x and the
increment of y.
We're going to look at this
a lot more in the future.
Let's look at the overhead.
So here we have this procedure,
and we're going to
look at how we do the
substitutions, the sequence of
substitutions.
Well, I'm going to try and add
together three and four.
Well, using the first rule
that I showed you, we
substitute three for x and four
four y in the body of
this procedure.
The body of the procedure is the
thing that begins with if
and finishes over here.
So what we get is, of course,
if three is zero, then the
result is four.
Otherwise, it's the sum of the
decrement of three and the
increment of four.
But I'm not going to worry about
these yet because three
is not zero.
So the answer is not four.
Therefore, this if reduces
to an evaluation of the
expression, the sum to the
decrement of three and the
increment of four.
Continuing with my evaluation,
the increment I presume to be
primitive, and so I
get a five there.
OK, and then the decrement
is also
primitive, and I get a two.
And so I change the problem
into a simpler problem.
Instead of adding three to four,
I'm adding two to five.
The reason why this is a simpler
problem is because I'm
counting down on x,
and eventually,
then, x will be zero.
So, so much for the
substitution rule.
In general, I'm not going to
write down intermediate steps
when using substitutions having
to do with ifs, because
they just expand things
to become complicated.
What we will be doing is saying,
oh, yes, the sum of
three and four results in the
sum of two and five and
reduces to the sum of two and
five, which, in fact, reduces
to the sum of one and six, which
reduces to the sum of
zero and seven over here, which
reduces to a seven.
That's what we're going
to be seeing.
Are there any questions for
the first segment yet?
Yes?
STUDENT: You're using one
plus and minus one plus.
Are those primitive
operations?
PROFESSOR: Yes.
One of the things you're going
to be seeing in this subject
is I'm going to, without
thinking about it, introduce
more and more primitive
operations.
There's presumably some large
library of primitive
operations somewhere.
But it doesn't matter that
they're primitive--
there may be some manual
that lists them all.
If I tell you what they do,
you say, oh, yes, I
know what they do.
So one of them is the
decrementor--
minus one plus-- and the other
operation is increment, which
is one plus.
Thank you.
That's the end of the
first segment.
[MUSIC PLAYING BY J.S. BACH]
PROFESSOR: Now that we have a
reasonably mechanical way of
understanding how a program
made out of procedures and
expressions evolves a process,
I'd like to develop some
intuition about how particular
programs evolve particular
processes, what the shapes of
programs have to be in order
to get particular shaped
processes.
This is a question about,
really, pre-visualizing.
That's a word from
photography.
I used to be interested in
photography a lot, and one of
the things you discover when
you start trying to learn
about photography is that you
say, gee, I'd like to be a
creative photographer.
Now, I know the rules, I push
buttons, and I adjust the
aperture and things like that.
But the key to being a creative
person, partly, is to
be able to do analysis
at some level.
To say, how do I know what it
is that I'm going to get on
the film before I
push the button.
Can I imagine in my mind the
resulting image very precisely
and clearly as a consequence of
the particular framing, of
the aperture I choose, of the
focus, and things like that?
That's part of the art of doing
this sort of thing.
And learning a lot
of that involves
things like test strips.
You take very simple images that
have varying degrees of
density in them, for example,
and examine what those look
like on a piece of paper when
you print them out.
You find out what is the range
of contrasts that you can
actually see.
And what, in a real scene,
would correspond to the
various levels and zones
that you have of
density in an image.
Well, today I want to look at
some very particular test
strips, and I suppose one of
them I see here is up on the
telestrator, so we should
switch to that.
There's a very important, very
important pair of programs for
understanding what's going on in
the evolution of a process
by the execution of a program.
What we have here are
two procedures
that are almost identical.
Almost no difference between
them at all.
It's a few characters that
distinguish them.
These are two ways of adding
numbers together.
The first one, which you see
here, the first one is the sum
of two numbers-- just
what we did before--
is, if the first one is
zero, it's the answer
of the second one.
Otherwise, it's the sum of the
decrement of the first and the
increment of the second.
And you may think of that
as having two piles.
And the way I'm adding these
numbers together to make a
third pile is by moving marbles
from one to the other.
Nothing more than that.
And eventually, when I run
out of one, then the
other is the sum.
However, the second procedure
here doesn't do it that way.
It says if the first number
is zero, then the
answer is the second.
Otherwise, it's the increment of
the sum of the decrement of
the first number
and the second.
So what this says is add
together the decrement of the
first number and the second-- a
simpler problem, no doubt--
and then change that result
to increment it.
And so this means that if you
think about this in terms of
piles, it means I'm holding
in my hand the
things to be added later.
And then I'm going
to add them in.
As I slowly decrease one pile to
zero, I've got what's left
here, and then I'm going
to add them back.
Two different ways of adding.
The nice thing about these two
programs is that they're
almost identical.
The only thing is where
I put the increment.
A couple of characters
moved around.
Now I want to understand the
kind of behavior we're going
to get from each of
these programs.
Just to get them firmly
in your mind--
I usually don't want to
be this careful--
but just to get them firmly in
your mind, I'm going to write
the programs again on the
blackboard, and then I'm going
to evolve a process.
And you're going to
see what happens.
We're going to look at the
shape of the process as a
consequence of the program.
So the program we started with
is this: the sum of x and y
says if x is zero, then
the result is y.
Otherwise, it's the sum of the
decrement of x and the
increment of y.
Now, supposing we wish to do
this addition of three and
four, the sum of three and
four, well, what is that?
It says that I have to
substitute the arguments for
the formal parameters
in the body.
I'm doing that in my mind.
And I say, oh, yes, three is
substituted for x, but three
is not zero, so I'm going to go
directly to this part and
write down the simplified
consequent here.
Because I'm really interested
in the behavior of addition.
Well, what is that?
That therefore turns into
the sum of two and five.
In other words, I've reduced
this problem to this problem.
Then I reduce this problem to
the sum of one and six, and
then, going around again
once, I get the
sum of zero and seven.
And that's one where x equals
zero so the result is y, and
so I write down here a seven.
So this is the behavior of the
process evolved by trying to
add together three and four
with this program.
For the other program, which
is over here, I will define
the sum of x and y.
And what is it?
If x is zero, then the result
is y-- almost the same--
otherwise the increment
of the sum of the
decrement of x and y.
No.
I don't have my balancer
in front of me.
OK, well, let's do it now.
The sum of three and four.
Well, this is actually a little
more interesting.
Of course, three is not zero as
before, so that results in
the increment of the sum of the
decrement of x, which is
two and four, which is
the increment of
the sum of one and--
whoops: the increment
of the increment.
What I have to do now is compute
what this means.
I have to evaluate this.
Or what that is, the result of
substituting two and four for
x and y here.
But that is the increment
of the sum of one
and four, which is--
well, now I have
to expand this.
Ah, but that's the increment
of the increment of the
increment of the sum
of zero and four.
Ah, but now I'm beginning
to find things I can do.
The increment of the increment
of the increment of-- well,
the sum of zero and
four is four.
The increment of four is five.
So this is the increment of the
increment of five, which
is the increment of six,
which is seven.
Two different ways of
computing sums.
Now, let's see.
These processes have very
different shapes.
I want you to feel
these shapes.
It's the feeling for the
shapes that matters.
What's some things we
can see about this?
Well, somehow this is
sort of straight.
It goes this way-- straight.
This right edge doesn't vary
particularly in size.
Whereas this one, I see that
this thing gets bigger and
then it gets smaller.
So I don't know what
that means yet,
but what are we seeing?
We're seeing here that somehow
these increments are expanding
out and then contracting back.
I'm building up a bunch
of them to do later.
I can't do them now.
There's things to be deferred.
Well, let's see.
I can imagine an abstract
machine.
There's some physical machine,
perhaps, that could be built
to do it, which, in fact,
executes these programs
exactly as I tell you,
substituting character strings
in like this.
Such a machine, the number of
such steps is an approximation
of the amount of
time it takes.
So this way is time.
And the width of the thing is
how much I have to remember in
order to continue the process.
And this much is space.
And what we see here is a
process that takes a time
which is proportional
to the argument x.
Because if I made x larger
by one, then I'd
had an extra line.
So this is a process which
is space-- sorry--
time.
The time of this process is
what we say order of x.
That means it is proportional
to x by some constant of
proportionality, and I'm not
particularly interested in
what the constant is.
The other thing we see here is
that the amount of space this
takes up is constant, it's
proportional to one.
So the space complexity of
this is order of one.
We have a name for
such a process.
Such a process is called
an iteration.
And what matters here is not
that some particular machine I
designed here and talked to
you about and called a
substitution machine
or whatever--
substitution model--
managed to do this in
constant space.
What really matters is this
tells us a bound.
Any machine could do this
in constant space.
This algorithm represented by
this procedure is executable
in constant space.
Now, of course, the model is
ignoring some things, standard
sorts of things.
Like numbers that are bigger
take up more space and so on.
But that's a level of
abstraction at which I'm
cutting off.
How do you represent numbers?
I'm considering every number
to be the same size.
And numbers grow slowly for the
amount of space they take
up and their size.
Now, this algorithm is different
in its complexity.
As we can see here, this
algorithm has a time
complexity which is also
proportional to the input
argument x.
That's because if I were to add
one to three, if I made a
larger problem, which is larger
by one here, then I'd
add a line at the top and I'd
add a line at the bottom.
And the fact that it's a
constant amount, like this is
twice as many lines as that,
is not interesting at the
level of detail I'm talking
about right now.
So this is a time complexity
order of the input argument x.
And space complexity, well,
this is more interesting.
I happen to have some overhead,
which you see over
here, which is constant
approximately.
Constant overhead.
But then I have something which
increases and decreases
and is proportional to
the input argument x.
The input argument x is three.
That's why there are three
deferred increments sitting
around here.
See?
So the space complexity
here is also order x.
And this kind of process, named
for the kind of process,
this is a recursion.
A linear recursion, I will call
it, because of the fact
that it's proportional to
the input argument in
both time and space.
This could have been
a linear iteration.
So then what's the essence
of this matter?
This matter isn't so obvious.
Maybe there are other models by
which we can describe the
differences between iterative
and recursive processes.
Because this is hard now.
Remember, we have-- those are
both recursive definitions.
What we're seeing there are both
recursive definitions,
definitions that refer to the
thing being defined in the
definition.
But they lead to different
shape processes.
There's nothing special about
the fact that the definition
is recursive that leads to
a recursive process.
OK.
Let's think of another model.
I'm going to talk to you
about bureaucracy.
Bureaucracy is sort
of interesting.
Here we see on a slide
an iteration.
An iteration is sort of
a fun kind of process.
Imagine that there's a
fellow called GJS--
that stands for me--
and he's got a problem:
he wants to add
together three and four.
This fella here wants to add
together three and four.
Well, the way he's going to do
it-- he's lazy-- is he's going
to find somebody else
to help him do it.
They way he finds someone
else to--
he finds someone else to help
him do it and says, well, give
me the answer to three
and four and return
the result to me.
He makes a little piece of paper
and says, here, here's a
piece of paper-- you go ahead
and solve this problem and
give the result back to me.
And this guy, of course,
is lazy, too.
He doesn't want to see this
piece of paper again.
He says, oh, yes, produce a new
problem, which is the sum
of two ad five, and return
the result back to GJS.
I don't want to see it again.
This guy does not want to
see this piece of paper.
And then this fellow makes a
new problem, which is the
addition of the sum of one and
six, and he give it to this
fella and says, produce that
answer and returned it to GJS.
And that produces a problem,
which is to add together zero
and seven, and give
the result to GJS.
This fella finally just says,
oh, yeah, the answer is seven,
and sends it back to GJS.
That's what an iteration is.
By contrast, a recursion
is a slightly
different kind of process.
This one involves more
bureaucracy.
It keeps more people busy.
It keeps more people employed.
Perhaps it's better
for that reason.
But here it is: I want
the answer to the
problem three and four.
So I make a piece of paper that
says, give the result
back to me.
Give it to this fella.
This fellow says, oh, yes, I
will remember that I have to
add later, and I want to get
the answer the problem two
plus four, give that one to
Harry, and have the results
sent back to me--
I'm Joe.
When the answer comes back from
Harry, which is a six, I
will then do the increment and
give that seven back to GJS.
So there are more pieces of
paper outstanding in the
recursive process than
the iteration.
There's another way to think
about what an iteration is and
the difference between an
iteration and a recursion.
You see, the question is, how
much stuff is under the table?
If I were to stop--
supposing I were to kill this
computer right now, OK?
And at this point I lose the
state of affairs, well, I
could continue the computation
from this point but everything
I need to continue the
computation is in the
valuables that were defined
in the procedure that the
programmer wrote for me.
An iteration is a system that
has all of its state in
explicit variables.
Whereas the recursion is
not quite the same.
If I were to lose this pile of
junk over here, and all I was
left with was the sum of one
and four, that's not enough
information to continue the
process of computing out the
seven from the original problem
of adding together
three of four.
Besides the information that's
in the variables of the formal
parameters of the program,
there is also information
under the table belonging to
the computer, which is what
things have been deferred
for later.
And, of course, there's a
physical analogy to this,
which is in differential
equations, for example, when
we talk about something
like drawing a circle.
Try to draw a circle, you make
that out of a differential
equation which says the change
in my state as a function of
my current state.
So if my current state
corresponds to particular
values of y and x, then I can
compute from them a derivative
which says how the state
must change.
And, in fact, you can see this
was a circle because if I
happen to be, say, at this place
over here, at one, zero,
for example, on this graph,
then it means that the
derivative of y is x, which
we see over here.
That's one, so I'm going up.
And the derivative of
x is minus y, which
means I'm going backwards.
I'm actually doing nothing at
this point, then I start going
backwards as y increases.
So that's how you
make a circle.
And the interesting thing to see
is a little program that
will draw a circle
by this method.
Actually, this won't draw a
circle because it's a forward
oil or integrator and
will eventually
spiral out and all that.
But it'll draw a circle
for a while
before it starts spiraling.
However, what we see here is two
state variables, x and y.
And there's an iteration that
says, in order to circle,
given an x and y, what I want
is to circle with the next
values of x and y being the old
value of x decrement by y
times dt where dt is the time
step and the old value of y
being implemented by x times dt,
giving me the new values
of x and y.
So now you have a feeling for
at least two different kinds
of processes that can
be evolved by
almost the same program.
And with a little bit of
perturbation analysis like
this, how you change a program
a little bit and see how the
process changes, that's how
we get some intuition.
Pretty soon we're going to use
that intuition to build big,
hairy, complicated systems.
Thank you.
[MUSIC PLAYING BY J.S. BACH]
PROFESSOR: Well, you've just
seen a simple perturbational
analysis of some programs.
I took a program that was very
similar to another program and
looked at them both and saw how
they evolved processes.
I want to show you some variety
by showing you some
other processes and shapes
they may have.
Again, we're going to take very
simple things, programs
that you wouldn't want
to ever write.
They would be probably the worst
way of computing some of
the things we're going
to compute.
But I'm just going to show you
these things for the purpose
of feeling out how to program
represents itself as the rule
for the evolution
of a process.
So let's consider a fun thing,
the Fibonacci numbers.
You probably know about
the Fibonacci numbers.
Somebody, I can't remember who,
was interested in the
growth of piles of rabbits.
And for some reason or other,
the piles of rabbits tend to
grow exponentially,
as we know.
And we have a nice model for
this process, is that we start
with two numbers,
zero and one.
And then every number
after this is the
sum of the two previous.
So we have here a one.
Then the sum of these
two is two.
The sum of those two is three.
The sum of those two is five.
The sum of those two is eight.
The sum of those two is 13.
This is 21.
34.
55.
Et cetera.
If we start numbering these
numbers, say this is the
zeroth one, the first one, the
second one, the third one, the
fourth one, et cetera.
This is the 10th one, the
10th Fibonacci number.
These numbers grow very fast.
Just like rabbits.
Why rabbits grow this way I'm
not going to hazard a guess.
Now, I'm going to try to write
for you the very simplest
program that computes
Fibonacci numbers.
What I want is a program that,
given an n, will produce for
me Fibonacci event.
OK?
I'll write it right here.
I want the Fibonacci of n, which
means the-- this is the
n, and this is Fibonacci of n.
And here's the story.
If n is less than two,
then the result is n.
Because that's what these are.
That's how you start it up.
Otherwise, the result is the sum
of Fib of n minus one and
the Fibonacci number,
n minus two.
So this is a very simple, direct
specification of the
description of Fibonacci numbers
that I gave you when I
introduced those numbers.
It represents the recurrence
relation in the simplest
possible way.
Now, how do we use
such a thing?
Let's draw this process.
Let's figure out
what this does.
Let's consider something very
simple by computing
Fibonacci of four.
To compute Fibonacci of
four, what do I do?
Well, it says I have--
it's not less than two.
Therefore it's the sum
of two things.
Well, in order to compute that
I have to compute, then,
Fibonacci of three and
Fibonacci of two.
In order to compute Fibonacci
of three, I have to compute
Fibonacci of two and
Fibonacci of one.
In order to compute Fibonacci
of two, I have to compute
Fibonacci of one and
Fibonacci of zero.
In order to compute Fibonacci
of one, well,
the answer is one.
That's from the base case
of this recursion.
And in order to compute
Fibonacci of one, well, that
answer is zero, from
the same base.
And here is a one.
And Fibonacci of two is really
the sum of Fibonacci of one.
And Fib of zero, in order to
compute that, I get a one, and
here I've got a zero.
I've built a tree.
Now, we can observe some
things about this tree.
We can see why this is an
extremely bad way to compute
Fibonacci numbers.
Because in order to compute
Fibonacci of four, I had to
compute Fibonacci of two's
sub-tree twice.
In fact, in order way to add one
more, supposing I want to
do Fibonacci of five, what I
really have to do then is
compute Fibonacci of four
plus Fibonacci of three.
But Fibonacci of three's
sub-tree has
already been built.
This is a prescription
for a process that's
exponential in time.
To add one, I have to multiply
by something because I take a
proportion of the existing thing
and add it to itself to
add one more step.
So this is a thing whose time
complexity is order of--
actually, it turns out
to be Fibonacci--
of n.
There's a thing that grows
exactly at Fibonacci numbers.
It's a horrible thing.
You wouldn't want to do it.
The reason why the time has to
grow that way is because we're
presuming in the model--
the substitution model that I
gave you, which I'm not doing
formally here, I sort of now
spit it out in a simple way--
but presuming that everything
is done sequentially.
That every one of these
nodes in this
tree has to be examined.
And so since the number of
nodes in this tree grows
exponentially, because I add a
proportion of the existing
nodes to the nodes I already
have to add one, then I know
I've got an exponential
explosion here.
Now, let's see if we can
think of how much
space this takes up.
Well, it's not so bad.
It depends on how much we have
to remember in order to
continue this thing running.
Well, that's not so hard.
It says, gee, in order to know
where I am in this tree, I
have to have a path
back to the root.
In other words, in order to--
let's consider the path I
would have to execute this.
I'd say, oh, yes, I'm going
to go down here.
I don't care which
direction I go.
I have to do this.
I have to then do this.
I have to traverse this tree
in a sort of funny way.
I'm going to walk this
nice little path.
I come back to here.
Well, I've got to remember where
I'm going to be next.
I've got to keep that in mind.
So I have to know
what I've done.
I have to know what's left.
In order to compute Fibonacci
of four, at some point I'm
going to have to be down here.
And I have to remember that I
have to go back and then go
back to here to do
an addition.
And then go back to here to do
an addition to something I
haven't touched yet.
The amount of space that
takes up is the
path, the longest path.
How long it is.
And that grows as n.
So the space--
because that's the length
of the deepest
line through the tree--
the space is order of n.
It's a pretty bad process.
Now, one thing I want to see
from this is a feeling of
what's going on here.
Why are there--
how is this program related
to this process?
Well, what are we seeing here?
There really are only
two sorts of things
this program does.
This program consists of
two rules, if you will.
One rule that says Fibonacci of
n is this sum that you see
over here, which is a node
that's shaped like this.
It says that I break up
something into two parts.
Under some condition over here
that n is greater than two,
then the node breaks
up into two parts.
Less than two.
No.
Greater than two.
Yes.
The other possibility is
that I have a reduction
that looks like this.
And that's this case.
If it's less than two, the
answer is n itself.
So what we're seeing here is
that the process that got
built locally at every place is
an instance of this rule.
Here's one instance
of the rule.
Here is another instance
of the rule.
And the reason why people think
of programming as being
hard, of course, is because
you're writing down a general
rule, which is going to be used
for lots of instances,
that a particular instance--
it's going to control each
particular instance for you.
You've got to write down
something that's a general in
terms of variables, and you
have to think of all the
things that could possibly fit
in those variables, and all
those have to lead to the
process you want to work.
Locally, you have to break up
your process into things that
can be represented in
terms of these very
specific local rules.
Well, let's see.
Fibonaccis are, of course,
not much fun.
Yes, they are.
You get something called the
golden ratio, and we may even
see a lot of that some time.
Well, let's talk about
another thing.
There's a famous game called the
Towers of Hanoi, because I
want to teach you how to think
about these recursively.
The problem is this one: I have
a bunch of disks, I have
a bunch of spikes, and it's
rumored that somewhere in the
Orient there is a 64-high tower,
and the job of various
monks or something is to move
these spikes in some
complicated pattern
so eventually--
these disks--
so eventually I moved all
of the disks from one
spike to the other.
And if it's 64 high, and it's
going to take two to the 64th
moves, then it's a long time.
They claim that the universe
ends when this is done.
Well, let's see.
The way in which you would
construct a recursive process
is by wishful thinking.
You have to believe.
So, the idea.
Supposing I want to move this
pile from here to here, from
spike one to spike two, well,
that's not so hard.
See, supposing somehow, by some
magic-- because I've got
a simpler problem--
I move a three-high
pile to here--
I can only move one
disk at a time, so
identifying how I did it.
But supposing I could do that,
well, then I could just pick
up this disk and move it here.
And now I have a
simple problem.
I have to move a three-high
tower to
here, which is no problem.
So by two moves of a three high
tower plus one move of a
single object, I can move the
tower from here to here.
Now, whether or not--
this is not obvious in any
deep way that this works.
And why?
Now, why is it the case that I
can presume, maybe, that I can
move the three-high tower?
Well, the answer is because I'm
always counting down, and
eventually I get down to
zero-high tower, and a
zero-high tower requires
no moves.
So let's write the algorithm
for that.
Very easy.
I'm going to label these towers
with numbers, but it
doesn't matter what they're
labelled with.
And the problem is to move an
n-high tower from a spike
called From to a spike called
To with a particular spike
called Spare.
That's what we're going to do.
Using the algorithm I informally
described to you,
move of a n-high tower from
From to To with a Spare.
Well, I've got two cases, and
this is a case analysis, just
like it is in all the other
things we've done.
If n is zero, then--
I'm going to put out
some answers--
Done, we'll say.
I don't know what that means.
Because we'll never use that
answer for anything.
We're going to do these moves.
Else.
I'm going to do a move.
Move a tower of height
less than n, the
decrement of n height.
Now, I'm going to move it
to the Spare tower.
The whole idea now is to move
this from here to here, to the
Spare tower-- so from
From to Spare--
using To as a spare tower.
Later, somewhere later, I'm
going to move that same n-high
tower, after I've done this.
Going to move that same n minus
one-high tower from the
Spare tower to the To
tower using the
From tower as my spare.
So the Spare tower to
the To tower using
the From as the spare.
All I have to do now is when
I've gotten it in this
condition, between these two
moves of a whole tower--
I've got it into that
condition--
now I just have to
move one disk.
So I'm going to say that some
things are printing a move and
I don't care how it works.
From the To.
Now, you see the reason why I'm
bringing this up at this
moment is this is an almost
identical program to this one
in some sense.
It's not computing the same
mathematical quantity, it's
not exactly the same tree, but
it's going to produce a tree.
The general way of making these
moves is going to lead
to an exponential tree.
Well, let's do this four-high.
I have my little crib sheet here
otherwise I get confused.
Well, what I'm going to put in
is the question of move a
tower of height four from one to
spike two using spike three
as a spare.
That's all I'm really
going to do.
You know, let's just do it.
I'm not going to worry
about writing out
the traits of this.
You can do that yourself because
it's very simple.
I'm going to move disk
one to disk three.
And how do I get to move
disk one to disk three?
How do I know that?
Well, I suppose I have to look
at the trace a little bit.
What am I doing here?
Well, and this is not--
n is not zero.
So I'm going to look
down here.
This is going to require
doing two moves.
I'm only going to look
at the first one.
It's going to require moving--
why do I have move tower?
It makes it harder
for me to move.
I'm going to move a three-high
tower from the from place,
which is four, to the
spare, which is two,
using three as my--
no, using from--
STUDENT: [INAUDIBLE PHRASE].
PROFESSOR: Yes.
I'm sorry.
From two--
from one to three using
two as my spare.
That's right.
And then there's another move
over here afterwards.
So now I say, oh, yes, that
requires me moving a two-high
tower from one to two using
three as a spare.
And so, are the same, and that's
going to require me
moving and one-high tower
from one to three
using two as a spare.
Well, and then there's lots of
other things to be done.
So I move my one-high tower from
one to three using two as
a spare, which I didn't
do anything with.
Well, this thing just proceeds
very simply.
I move this from one to two.
And I move this disk
from three to two.
And I don't really want
to do it, but I
move from one to three.
Then I move two to one.
Then I move two to three.
Then one to three.
One to two.
Three to two.
Three to one.
This all got worked out
beforehand, of course.
Two to one.
Three to two.
One to three.
STUDENT: [INAUDIBLE PHRASE].
PROFESSOR: Oh, one to three.
Excuse me.
Thank you.
One to two.
And then three to two.
Whew.
Now what I'd like you to think
about, you just saw a
recursive algorithm for doing
this, and it takes exponential
time, of course.
Now, I don't know if there's
any algorithm that doesn't
take exponential time--
it has to.
As I'm doing one operation--
I can only move one
thing at a time--
there's no algorithm
that's not going to
take exponential time.
But can you write an iterative
algorithm rather than a
recursive algorithm
for doing this?
One of the sort of little things
I like to think about.
Can you write one that, in
fact, doesn't break this
problem into two sub-problems
the way I described, but
rather proceeds a step at a time
using a more local rule?
That might be fun.
Thank you so much for
the third segment.
Are there questions?
STUDENT: [INAUDIBLE] a way to
reduce a tree or recursion
problem, how do you save the
immediate work you have done
in computing the Fibonacci
number?
PROFESSOR: Oh, well, in fact,
one of the ways to do is what
you just said.
You said, I save the
intermediate work.
OK?
Well, let me tell you--
this, again, we'll see later--
but suppose it's the case that
anytime I compute anything,
any one of these Fibonacci
numbers, I remember the table
that takes only linear time
to look up the answer.
Then if I ever see it again,
instead of doing the
expansional tree,
I look it up.
I've just transformed my problem
into a problem that's
much simpler.
Now, of course, there are the
way to do this, as well.
That one's called memoization,
and you'll see it sometime
later in this term.
But I suppose there's a very
simple linear time, and, in
fact, iterative model for
computing Fibonaccis, and
that's another thing you should
sit down and work out.
That's important.
It's important to see
how to do this.
I want you to practice.
