The following
content is provided
under a Creative
Commons license.
Your support will help MIT
OpenCourseWare continue
to offer high quality
educational resources for free.
To make a donation or
view additional materials
from hundreds of MIT courses,
visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: If you guys
want me to cover anything
in particular, is
there anything you
didn't understand in lecture?
In the last section, I
covered the recursion trees
because they will
be on the Pset,
and people said they
were a bit unclear,
so we can do that and cover less
of the stuff that I have here.
Or if there's anything else,
you can tell me what you want.
So there I cover recursion
trees because someone said, hey,
can you go over that again?
Is there any pain points?
No?
OK.
So then I'm going to
give you the same choice
that I gave to people
last time, and that
is we can go over
recursion trees again,
but if I do that, then I
won't have time to go over
the code for deleting a node
from a binary search tree.
So we'll go through
the theory and you guys
will have to go through
the code on your own.
But instead, we'll go
over recursion trees again
and remember how you
solve a recurrence using
recursion trees.
The alternative is
we don't do that
and we complete
the deletions part.
AUDIENCE: I feel like
covering deletions,
since we didn't do that in
lecture, that would probably
be more helpful.
PROFESSOR: Let's take a vote.
Who wants to do deletions
in painstaking detail?
So deletions and not recursion?
Who wants to do recursion
trees and not deletion?
AUDIENCE: It's about equal.
PROFESSOR: It's equal
and nobody cares.
I'm really sad.
AUDIENCE: Let's do
both in half detail.
PROFESSOR: OK, sure.
Who remembers merge sort?
What does merge sort
do really quick?
AUDIENCE: It takes some
sort of unsorted array,
splits it in half, and
then continually splits it,
and then once it finally
gets to the point
where you have arrays of two
elements, then it sorts them,
and then sorts those,
and then sorts those.
It's a fun thing.
And then it merges [INAUDIBLE].
PROFESSOR: That's so much code.
I don't like to write much code
because for every line of code
that you write, you
might have a bug in it,
so I like to write less code.
So the way I do it is when I
get to an array size of one
element, I know
it's already sorted.
So merge sort.
You have an array,
it's unsorted.
Split it into two halves,
call merge sort on each half,
assume that magically, they're
going to come back sorted,
and then you merge
the sorted halves.
How much time does merging take?
OK.
So the recursion for the
running time of merge sort?
AUDIENCE: Why does
it take n time?
Just too large?
AUDIENCE: Isn't it
the finger thing where
you take each element,
and you're like, this one,
is that greater or less than,
then you put it in the array.
So you get--
PROFESSOR: Please take my
word for it that it's order n.
AUDIENCE: I'll explain it
and then I'll be confused.
PROFESSOR: OK, so order n.
What's the recursion?
Don't give me the
solution because then I
can't do the trees anymore,
so give me the recursion
before it's solved.
Give me the recurrence formula.
So it starts with T of N, right?
AUDIENCE: It starts with
N over 2 plus N, I think.
PROFESSOR: Perfect.
So you take the array,
you split it into two,
you call merge sort on the
two halves of the arrays.
So you call merge sort twice.
That's why you have a 2 here.
The 2 matters.
Without it, you get
a different answer.
And when you call it,
the arrays that you
give it are half the
size, and then merge
takes order and time.
Splitting depends on what you're
using to store your arrays.
Can be constant time or it
can be order N. So the time
won't change because of split.
How do we solve this recurrence?
The recursion tree
method says that we're
going to draw a call graph.
So we start out
with a call to merge
sort with an array
of size N. Then
it's going to call
merge sort again,
but after the array is split.
So it's going to call merge
sort twice, size is N over 2.
This guy gets an array of
N over 2, calls merge sort.
Two arrays, sizes
N over 4, N over 4.
This does the same.
So this goes on forever and ever
and ever until at some point
we reach our base case.
So we're going to
have a bunch of calls
here where the array size is?
What's our base case?
1.
Excellent.
So this is the call graph
for merge sort, and let's
put the base case here so we
know what we're talking about.
T of 1 is theta 1.
Now inside the
nodes, we're going
to put the cost for each
call without counting
the sub-call, so
the children here.
That's this guy here,
except instead of order N,
I will write CN.
Remember how sometimes
we use CN instead
of the order of notation?
The reason we do that is if I
put in the asymptotic notation,
then we're going to be
tempted to sum them up.
You're allowed to sum terms
using asymptotic notation as
long as there's a
finite number of them,
but here, it turns out there's
an infinite number of them.
Also, if you go this way,
you can never go wrong.
You always get the
right answer, so that's
why we switch from
order N to CN.
In order to merge sort
an array of size N,
we're going to merge sort
two arrays of size N over 2
and then spend CN time
on doing the merge.
What are the costs here?
To sort an array
of N over 2, what's
the cost outside
the cost to merge?
AUDIENCE: C of N over 2.
PROFESSOR: Perfect.
C times N over 2.
C times N over 2.
How about here?
AUDIENCE: C times N over 4.
PROFESSOR: Perfect.
CN over 4.
My nodes are really ugly.
I should have drawn them
like this from the beginning.
CN over 4.
There you go.
How about down here?
AUDIENCE: C of N
over 2 to the i.
PROFESSOR: You're
going on step ahead.
We'll do that right next.
AUDIENCE: C of N
over log N, right?
Because they're
log N levels, so--
PROFESSOR: Let's not worry
about the number of levels.
You're ruining my steps.
I was going to get to
that two steps after this.
AUDIENCE: Is it just C?
PROFESSOR: Yep.
So array size is 1, right?
So the cost is C. C, C,
C, C. OK, you guys got it
if you're thinking
of levels already.
The next thing I
want to do is I want
to figure out how many
levels I have in this tree.
Why do I care about that?
The answer for T of N is the
sum of all these costs in here
because the cost of merge
sorting an array of size N
is the merge sort plus the costs
for sorting the two arrays.
And the nodes here keep
track of all the time spent
in recursive sub-calls, so if
we can add up everything up,
we have the answer to T of
N. It turns out the easiest
way to do that is to sum
up the cost at each level
because the costs are
this guy copied over here.
For a level, they
tend to be the same,
so it's reasonably
easy to add them up,
except in order to be
able to add those up,
you have to know how
many levels you have.
So how do I know how
many levels I have?
Someone already told me log
N. How do I get to that log N?
So when I get to the
bottommost level,
the number has to be 1, the
number next to the node,
because that's my base case.
When I have a one element
array, it's sorted, I'm done.
I return.
So I can say that
for each level,
the number next to the node is
something as a function of L.
Here, I'm going to
say that this is
N over 1, which is N
over 2 to the 0 power.
And this is N over 2, so it's
N over 2 to the first power.
This is N over 2 to the
second, and so on and so forth.
It might not be obvious if
you only have two levels.
I don't want to draw
a lot on the board
because I don't have a lot
of space and I'd get my nodes
all messed into each other.
If it takes more than two levels
to see the pattern, go for it.
Expand for three levels,
four levels, five levels,
whatever it takes to get it
right on a Pset or on a test.
So you see the pattern,
then you write the formula
for the node size at the level.
And assuming this
pattern holds, we
see that the size of a
node at level l, the size
is 2 N over 2 to the l minus 1.
Fair enough?
You can say N over 2
to the l, and forget
that there's a minus 1, and then
the asymptotics will save you,
so it's no big deal, but
this is the real number.
So that means that at the
bottommost level, at level l,
this size is going to be 1.
N over 2 to the l
minus 1 equals 1.
So now this is an equation,
so I can solve for l.
I pull this on the right side,
N equals 2 to the l minus 1,
so l minus 1 equals-- anyone?
The inverse of an exponential?
AUDIENCE: I wasn't
paying attention.
Sorry.
AUDIENCE: Log N.
PROFESSOR: The inverse of an
exponential is a logarithm.
Keep that in mind for
solving 6.006 problems.
l minus 1 is log N so l is
log n plus 1, roughly log n.
I could use log n plus 1
and go through the math.
It's a bit more painful
and, because we're
using asymptotics, it
doesn't really matter.
So now we know how
many levels we have.
Let's see what's the
cost at the level.
So all the calls
at a certain level,
what's the sum of the costs?
For this level, what's the cost?
CN.
And That was the easy question.
Just the root, right?
How about this level?
Because I have two nodes, the
cost in each node is CN over 2.
How about this level?
Four levels, each
level CN over 4.
How about the bottom level?
AUDIENCE: CN.
PROFESSOR: Why is it CN?
AUDIENCE: Because there
are N arrays of size 1.
PROFESSOR: N arrays of size 1.
Excellent.
A cute argument I heard once
is you start out with N,
you split it into N
over 2 and N over 2.
Then you split this guy
in N over 4, N over 4,
so this is like
conservation of mass.
If you start with N and here,
you don't end up with N total,
then you lost some element
somewhere on the way.
So CN.
CN, CN, CN, CN.
I think I see a pattern.
I think it's reasonable to say
that for every level, it's CN.
And if you write the
proof, you can prove that
by using math instead
of waving hands.
So CN times the number
of levels, right?
The answer for this guy
is C of N is CN times l.
What's l?
AUDIENCE: N log N.
PROFESSOR: Roughly.
OK So order of N
log N. C becomes
order of, l is order of
log N, N stays the same.
Any questions?
Are people getting it or
did I confuse you even more?
AUDIENCE: We got it.
PROFESSOR: OK, sweet.
Thank you for the encouragement.
So this gets you through
problem one of Pset 2.
So in this case, the
tree is nicely balanced.
The cost at each
level is the same.
When [INAUDIBLE] talked about
recursion trees in lectures,
he showed two more
trees, one where
pretty much all the cost was up
here-- the cost of the children
was negligible-- and one
tree where all the cost was
concentrated here, so the
cost of all the inner nodes
was negligible and the leaves
were doing all the real work.
So don't be scared if your
costs aren't the same.
Just sum them up and you'll
get to the right answer.
Now I'm going to talk
about binary search trees,
except I will make a five
minute general talk about data
structures before I do that.
So we use the term
"data structures."
I think we covered
it well, and I
want to give you
a couple of tips
for dealing with them on Pset 1.
A data structure is
a bunch of algorithms
that help you store and
then retrieve information.
You have two types
of algorithms.
You have queries,
and you have updates.
You start out with an
empty data structure,
like an empty binary search
tree or an empty list,
and then you throw
some data at it.
That's when you update it.
Then you ask it some questions,
and that's when you query it.
Then maybe you throw more data
at it, so you do more updates,
and you ask more questions,
so you do more queries.
What are the queries
and the updates
for the binary search trees
that we talked about in lecture?
AUDIENCE: A query
would be like, what's
your right child,
what's your left child?
PROFESSOR: So that's for a node.
AUDIENCE: What are
you looking for?
PROFESSOR: I'm
looking for something
for the entire infrastructure.
So for the entire
tree, what's a question
that you would ask the tree?
PROFESSOR: Max.
PROFESSOR: OK.
Min.
AUDIENCE: Next larger.
PROFESSOR: Next larger.
Are you looking at the nodes?
AUDIENCE: Is there an are
you balanced question?
PROFESSOR: Well, I would say
that the most popular operation
in a binary search
tree is Search, which
looks for-- we call
it Find in the code
because most code
implementations call it Find
nowadays, but
binary search tree.
What are you going to do in it?
You search for a value.
That's why it has the
Search in binary search.
So queries are operations where
you ask questions to the data
structure and it doesn't change.
How about updates?
What did we learn for updates?
AUDIENCE: Insert.
PROFESSOR: Excellent.
So Insert was
covered in lecture,
and we're doing Delete today.
So data structures
have this property
that's called the
representation invariant, RI,
or Rep Invariant.
Actually, before I get
there, the rep invariant
says that the data in
the data structures
is organized in this
way, and as long
as it's organized in this way,
the data structure functions
correctly.
Can someone guess
for a sorted array
what's the
representation invariant?
AUDIENCE: It can mean sorted.
PROFESSOR: Yeah.
A sorted array should be sorted.
Sounds like a very
good rep invariant.
So the elements should
be stored an array.
Every element should be smaller
than any element after it.
And as long as the rep
invariant holds, so as long
as elements are stored in
the right way in the data
structure, the queries will
return the right results.
If the rep invariant
doesn't hold,
then God knows what's
going to happen.
What can you do
in a storage array
as long as the rep
invariant holds?
Sorted array.
What's the reason why I
would have a sorted array?
What can I do that's
fast in a sorted array?
AUDIENCE: Min and Max.
PROFESSOR: I can
do that very fast.
That's good.
What's the running time?
AUDIENCE: A constant.
PROFESSOR: Perfect.
Min you look at the beginning,
Max you look at the end.
Yes?
AUDIENCE: Binary search.
PROFESSOR: Binary search.
That's the other
reason for that.
So binary search
runs in order log
N time, doesn't have to
look at most of the array,
tells you whether an
element is there are not.
Now, what if the
array is unsorted?
Will the algorithm work?
It might say
something isn't there
when it actually is there.
You can do binary search
on a non-sorted array.
So if the rep
invariant doesn't hold,
your queries might give
you a wrong answer.
How about updates?
How do you search
something in a sorted list?
AUDIENCE: You find
where it should go
and you move everything.
PROFESSOR: Yep.
So you have to move
everything, make room for it,
and put it there so
that the array is still
sorted at the end.
You can't just append
things at the end,
even though that would be
faster and lazier and less code.
When you do an update
to a data structure,
you have to make sure that
the rep invariant still
holds at the end.
Sort of a correctness proof
for an update algorithm
says that if the rep invariant
holds at the beginning,
the rep invariant is
guaranteed to hold at the end.
Why do we care about
this rep invariant stuff?
Suppose you have a problem,
say on the next Pset, that
asks you to find the place
that's slow in your code
and then speed it up.
And suppose you recognize
the data structure there,
and you say that's
inefficient, and you
want to implement another
data structure that
would be more efficient.
You're going to implement it.
You might have
bugs in an update.
How do you find the bugs?
Queries give you
the wrong answers.
You might do queries a long
time after you do updates,
and you're not going to
know which update failed.
The right way to do this
is you implement the method
called Check RI-- that's
what I call it-- so
check the representation
invariant.
And that method walks through
the entire data structure
and make sure that the
rep invariant holds,
and if it doesn't, it
raises an exception
because you know that whatever
you try to do from there
is not going to work, so
there's no reason to keep going.
So at the end of every
update, you add a call
to this Check RI
method until you're
sure that your code is correct.
And after you're done
debugging your code,
you remove this method
and you submit the code.
Why do I want to
remove the method?
It might be painfully slow
and inefficient, much slower
than the actual
queries and updates.
For example, let's take a heap.
Do people remember
heaps from lecture?
What's the query for a heap?
Say you have a max heap.
What's a query?
AUDIENCE: Where's the max?
PROFESSOR: OK, cool.
So for a max heap, a
query would be max.
Running time?
AUDIENCE: Constant.
PROFESSOR: Perfect.
Constant.
What do you do?
Look at the top?
AUDIENCE: Yeah, exactly.
PROFESSOR: OK.
Sweet.
So what are the two popular
updates in a max heap?
AUDIENCE: There would
be Insert as well.
PROFESSOR: OK.
Insert.
And did we teach
you general delete?
Usually Extract Max is simpler.
That's all you need.
What's the running
time for Insert?
Do people remember heaps?
AUDIENCE: I think it was per
N, but I'm not completely sure.
PROFESSOR: Anyone else?
It's not.
Life would be bad
if it would be N.
AUDIENCE: N squared?
PROFESSOR: No.
It's better than N, so you
guys are doing a binary search
over the few running times
that I gave you earlier.
AUDIENCE: [INAUDIBLE]
add to the N,
and then you compare your
neighbor, and then you
[INAUDIBLE].
AUDIENCE: If it's an
array, there isn't--
PROFESSOR: So conceptually,
a heap looks like this.
And yeah, it becomes
an array eventually,
but let's look at it this way.
It is a full binary tree.
Binary tree means that each
node has at most two children,
and full means that every
level except for the last level
is completely populated.
So every internal node
has exactly two children,
and in here, every node
except for some nodes
and then some nodes
after it will not have.
Everything to the left
is fully populated,
and then at some point,
you stop having children.
It turns out that this is
easy to store in an array,
but I will not go over that.
Instead, I want to
go over inserting.
What's the rep
invariant for a heap?
AUDIENCE: The max
in the top, right?
Well, for max heap, and
then the two children
are less than the next node.
PROFESSOR: All right.
So the guy here has to be
bigger than these guys,
then the guy here has to
be bigger than these guys,
and so on and so forth.
And if you use
induction, you can
prove that if this
is bigger than this,
it has to be bigger than
these guys, and bigger
than these guys, and bigger
than everything, and it's a max.
That's the reason why we
have that rep invariant.
So the way we insert
a node is we add it
at the bottom, the only
place where we could add it.
And then if this guy is
bigger than this guy,
the rep invariant is
violated, so we swap them
in order to fix that.
Now the guy is here.
If this is bigger than
this, we do another swap.
If this is bigger than
this, we do another swap.
So you're going to go from the
bottom of the heap potentially
all the way up to the root.
So the running time
of insert is order
of the height of the heap.
Now, the heap is a
full binary tree.
I said "full."
I keep saying "full."
The reason I care about full
is that the full binary tree
is guaranteed to have a
height of log N. It's always
log N, where N is
the number of nodes.
So inserting in a
heap takes log N.
AUDIENCE: I have a question.
Didn't they say that
because it's in an array,
then to find it-- oh no, I
guess because you can still
do the swaps.
PROFESSOR: You can
still do the swaps
when you have it
serialized in an array.
You know that given
an item's index,
the parent is that
index divided by 2.
So you add an element
at the end of the array,
and then you know
what the parent is,
and then you keep swapping
and swapping and swapping
towards the [INAUDIBLE].
AUDIENCE: You don't
ever have to put it in
and shift everything over.
You're only swapping.
PROFESSOR: Yep.
You only swap.
That's important.
Thanks for asking.
That's important.
So log N. Extract max,
take my word for it,
also log N. What's the
running time for checking
the invariant in a heap?
So to make sure that this guy is
a heap, if I had numbers here,
what would you have to do?
AUDIENCE: You'd have
to look at every node.
PROFESSOR: Yep.
So running time?
AUDIENCE: Theta of N.
PROFESSOR: Yep.
So if I'm going to
submit code for a heap
where the operations
are our order of log N,
or order 1, but then each
of these calls Check RI,
that's going to be
painfully slow because I'm
making the updates be
order N instead of log N.
So you're putting Check
RI calls in every update.
You debug your code.
When you make sure it's
correct, you remove those,
and then you submit the Pset.
Make sense?
Sweet.
And we looked a little bit
at heaps, which is good.
Binary search trees.
So a binary tree is a
tree where every node
has at most two children.
When we code this up, we
represent a node as a Python
object, and for a node, we
keep track of the left child,
of the right child, parent,
and then this is a hollow tree.
It's not very useful.
This becomes useful when you
start putting keys in the nodes
so that you can find them and
do other things with them.
So each node has a key.
Let me draw a
binary search tree.
Can people see this?
So this is a binary tree.
Can someone say something a
bit more specific about it?
AUDIENCE: It's unbalanced.
PROFESSOR: OK.
It's imbalanced.
So that means that
finding things all the way
at the bottom is
going to be expensive.
What else?
So I said it's a binary tree.
Give me something more specific.
So binary tree just means that
every node has two children.
There's a bit more
structure in this guy.
So if I look at the
root, if I look at 23,
all the nodes to the
left are smaller.
All the nodes to the
right are bigger.
Now, if I look at 8, all the
nodes to the left are smaller,
all the nodes to the
right are greater.
This additional rep invariant
defines a binary search tree.
This is what we
talked about in class.
BST.
Why would I want to
have this rep invariant?
It sounds like a pain
to maintain nodes
with all these
ordering constraints.
What's the advantage
of doing that?
AUDIENCE: Search is fast.
PROFESSOR: Yep.
Search is fast.
How do I do search?
If you're looking
for 42 or for 16,
you'd be like, oh,
it's less than 23.
I'll get on this path.
PROFESSOR: So start at the root,
compare my key to the root.
If it's smaller, go left.
If it's bigger, go right.
Then keep doing that
until I arrive somewhere
or until I arrive at a dead
end if I'm looking for 14.
This is a lot like
binary search.
Binary search in an array,
you look at the middle.
If your key is smaller, go left.
If your key is
bigger, then go right.
Let's look at the
code for a little bit.
Look at the BST Node
Class, and you'll
see that it has the fields
that we have up here.
And look at the Find method,
and this is pretty much
the binary search code.
Lines 8 and 9 have the return
condition when you're happy
and you found the key,
and then line 10 compares
the key that you're looking
for with the key in the node
that you're at, and then lines
11, 14, 16, and 19 are pretty
much copy pasted,
except one of them
deals with the left
case, the other one
deals with the right case.
What is the running
time for Find?
AUDIENCE: Wouldn't
it be log N, right?
PROFESSOR: I wish.
If this is all you have
to do to get log N,
then I would have to
write a lot less code.
So not quite log N. We will
have to go through next lecture
to get to log N. Until then,
what's the running time?
AUDIENCE: Order h.
PROFESSOR: Yep.
So you told me at the
beginning it's unbalanced.
AUDIENCE: Yeah.
PROFESSOR: So then it's
not going to be fast.
OK, so order h.
The reason why we care
about h, and the reason
we don't say order
N, is because next
lecture after we learn
how to balance a tree,
there's some magic that you can
do to these binary search trees
to guarantee that the
height is order of log N.
And then we'll go through all
the running times that we have
and replace h with log N.
Now, it happens that in this
case, if you would have told
me order N, I couldn't argue
with you because worst case,
searches are order N. Can
someone give me a binary search
tree that exposes
this degenerate case?
Yes?
AUDIENCE: If it's
completely unbalanced
and every node is greater
than the parent nodes.
PROFESSOR: So give me some
inserts that create it.
AUDIENCE: Insert 5.
PROFESSOR: 5.
AUDIENCE: Insert 10.
PROFESSOR: 10.
AUDIENCE: Insert 15.
PROFESSOR: 15.
AUDIENCE: Insert 20.
PROFESSOR: Yep.
And I could keep going.
I could say, 1, 2, 3, 4, 5.
I could say 5, 10, 15.
As long as these keep
growing, this is basically
going to be a list,
so searching is
order N. This is
a degenerate case.
Turns out it doesn't happen
too often in practice.
If you have random data, the
height will be roughly log N.
But in order to avoid
those degenerate cases,
we'll be doing balanced
trees later on.
So we covered Find.
We know it's order h.
How do you insert,
really quickly?
AUDIENCE: Do you
mean in searching
when it's balanced
or unbalanced?
PROFESSOR: This guy.
So the trees look
exactly the same.
If it's balanced, it's
going to look more
like that than like this.
Actually, this is balanced.
This is perfectly unbalanced.
This is somewhere in the middle.
If it's balanced, it's just
going to look more like this,
but it's still a
binary search tree.
How would you insert a node?
Yes?
AUDIENCE: Can't you
start at the root
and find your way down, and
then the first open child
that you see that's applicable
to your element, state it then?
PROFESSOR: Yep.
So if I wanted to insert
14, which way do I go?
AUDIENCE: So you'd look
at 23, and you'd say,
it's less than 23, go left.
You'd look at 8.
You'd say, it's greater than 8.
You'd go right.
Look at 16.
You'd say it's less,
so you go left.
15, it's less.
Then you have an open spot
so you stick it there.
PROFESSOR: Excellent.
Thank you.
Yes?
AUDIENCE: I have a question.
What if we want to insert 5?
Then--
PROFESSOR: So if you
want to insert who?
AUDIENCE: 5.
Or actually no, we can't.
I'm thinking, is there any case
in which need to move a node?
PROFESSOR: How
would you insert 5?
Let's see.
What would you do for 5?
AUDIENCE: For 5, then we'd
insert it to the right of 4,
right?
PROFESSOR: Smaller,
smaller, greater, 5.
Right?
AUDIENCE: So there would
be no case in which we'd
need to swap nodes or something?
PROFESSOR: No.
You're thinking ahead.
We'll talk about that a little
later when we get to deleting.
As long as you follow a path in
the tree, the path that finding
would get you to, as soon
as you hit a dead end,
that's where your node belongs.
Because you know next time
you're going to search for it,
the search is going to follow
that path and find the node.
Yes?
AUDIENCE: If you have values
are the same, like two
nodes at the same
number, does it
matter which side you put it on?
PROFESSOR: You don't.
AUDIENCE: Oh, I see.
It's more like you would only
have four 1's in the tree.
PROFESSOR: Yes.
So if you're trying to
store keys and values,
then what you'd
have to do if you
want to allow multiple
values for the same key
is you have a linked
list going off
of this, which node
becomes an array of values
aside from the key.
Smart question.
Thank you.
That trips you up every
time you do actual code,
so that's the right
question to ask yourself
when you're implementing this.
Will I have duplicates?
How do I handle them?
We don't.
We take the easy way out.
So if you look at
Insert, on the next page,
you will see that the code
is pretty much the Find code
copy pasted, except
when Self Left is None
or Self Right is None,
instead of returning,
it creates a new node.
Does that make sense to people?
All right.
So Delete is going to be the
hardest operation for today.
Before we do that, let's
do a warm up operation.
Let's say I want to implement
Find Next Larger, also called
Successor in some
implementations.
So I have a node.
Say I have node 8, and I
want to find the next key
in the tree that's
strictly larger than 8
but smaller than anything else.
So if I would take these nodes
and write them down in order,
I want to find the element
that would go right after it.
How do I do that?
Don't cheat.
Don't look at the code, or make
my life easier and do searches.
AUDIENCE: Go down
one to the right,
and you try to get down
left as far as you can.
PROFESSOR: OK.
Very good.
So I have a node, and it
has some subtree here,
so I can go to the right here,
I can go all the way left.
We have an operation
that does this,
and it's called Min for a tree.
In order to find the minimum
in a binary search tree,
you keep going left.
For example, in this case,
you get 4, which is good.
So the way you would code
this up is if you have Min,
you go to the right
if you can, and then
you call Min on the subtree.
And you can see that lines
3 and 4 do exactly that.
Good guess.
But you can line
1 says case one,
so you have the right
answer for one case.
Now we have to handle
more difficult cases.
What if instead, I go
down a bunch of nodes,
and I want to find the successor
for this guy, for example,
and there's nothing here.
What do I do?
So if I want to find the
successor for 8, what do I do?
Sorry.
It has an answer.
What if I want to find
the successor for 4?
AUDIENCE: Go up one.
PROFESSOR: OK.
Go up one.
Why does that work?
AUDIENCE: You know it's
going to be greater.
PROFESSOR: So I'm
going up right.
So I know that everything here
is guaranteed to be smaller,
everything here is guaranteed
to be greater than this guy.
This guy is up right, so this
is guaranteed to be greater
than this, and
everything here is
guaranteed to be greater than
this, and so on and so forth
for the entire tree.
So if I go up right, I'm happy.
I definitely found my answer.
Now, what if I have something
that looks like this,
and I want to find the
successor for this guy?
AUDIENCE: There is none.
PROFESSOR: In this
case, there is none
if there's nothing else here.
What if I have this,
but then I have this?
So I came down this way.
AUDIENCE: Are you saying you're
calling on that last node?
PROFESSOR: Yep.
AUDIENCE: Find the larger?
I guess you'd just
trace back up.
PROFESSOR: And where do I stop?
AUDIENCE: It affects the tree,
so you go up one from there.
You don't stop there.
PROFESSOR: Why
can't I stop here?
AUDIENCE: Because you know
that that-- not necessarily.
AUDIENCE: You know that
everything in that long branch
right there is less than
that node [INAUDIBLE].
PROFESSOR: This is to
the left of this guy,
so this guy has to be
greater than everything here,
and then you can repeat the
argument that we had before.
So here, we could
stop right away
because we could branch left.
In this case, you have
to go up until you're
able to go left and up.
If you get to the root,
then what happened?
Then we're in this case,
and you have no successor.
So take a look at the code.
The next larger,
lines 1 through 9.
Case two, 6 through
8, does exactly that.
If I can't go to my right
and find the tree there,
then I go up through
my parent chain,
and as long as I have
to go up to the left,
so as long as I'm the
right child of my parent,
I have to keep going.
The moment I find the parent
where I'm the left child,
I stop.
That's my successor.
What if I would have to find
the predecessor instead?
So the element that's
smaller than me
but bigger than everything
else in the tree.
What would I do?
AUDIENCE: It's
just the opposite.
PROFESSOR: Just the opposite.
So how do I do the opposite?
AUDIENCE: You can take the
max of the left side tree,
or traverse up, and
if that's less than--
PROFESSOR: OK, so if I
have a left subtree, fine.
Call max on it and get
the rightmost node there.
If not, I go up,
and when do I stop?
When I go left or right?
AUDIENCE: You'd
have to go right.
Is that right?
PROFESSOR: Yep.
So last time, in this
case, when I was going up,
if I was going left,
I had to keep going,
and the moment I went right,
I was happy and I stopped.
What if I want to
find the predecessor?
It's the opposite, right?
So I will go this way, and
the moment I can go this way,
I'm done.
How do you do this in code?
Slightly tricky.
Just slightly, I promise.
AUDIENCE: [INAUDIBLE].
PROFESSOR: It's hard.
What I would do is
copy paste the code,
replace "left" with "right"
everywhere, and replace "min"
with "max."
You get it done.
So we talked about how the
tree is symmetric, right?
So every time, instead of
saying "left," you say "right,"
and instead of saying
"min," you say "max."
That's how you do this.
How do we do deletions?
So suppose I'm in this tree
and I want to delete 15.
What do I do?
AUDIENCE: Kill it.
PROFESSOR: Kill it.
Very good.
What if I want to delete 16?
What do I do?
AUDIENCE: You need to
put 15 where 16 is.
PROFESSOR: OK.
So I would put 15 here.
So I had 16.
Suppose I have a big tree here.
Actually, let's go
for an easier case.
Let's say I have this tree here.
So you're here, you
have a big tree here,
you don't have
anything here, and you
want to delete this guy.
AUDIENCE: You know
that everything less
than the top node is
going to be less than it,
so you can just move that up.
PROFESSOR: Everything
less than this guy is also
going to be less than this guy.
So you're saying move
the whole tree up.
AUDIENCE: Yep.
PROFESSOR: So the
way we do that is
we'd take this node's left
link and make it point here,
and take this guy's parent
link and make it point here,
and this guy sort of goes away.
So we have two
cases for deleting.
We have if you're a
leaf, we'll take you out.
Sorry.
I got confused.
If you have one
child and that child
is in the same direction as your
parent, then you can do this.
What if you have one child,
but it's a zigzag like this?
What do you do?
AUDIENCE: It's still greater
than, so you do the same thing.
PROFESSOR: Exactly.
Same thing.
Just change this guy, change
this guy, and I'm happy.
So it doesn't matter if you have
a zigzag or a straight line.
It might help you think
about it to convince yourself
that the code is correct, but in
the end, you do the same thing.
Now, what if I want
to delete node 8?
So what if I have a nasty case
where I want to delete this guy
and it has children both on
the left and on the right?
AUDIENCE: You have to take
8, compare it to its parent
and compare it to
its right child,
and see which one
is greater in order
to figure out which node
gets replaced in its spot.
PROFESSOR: OK.
So there is replacing
that's going to happen.
The answer is really tricky.
I always forget
this when coding.
Try to understand it,
and if it doesn't work,
refer to the textbook.
When you forget it, because
you will, refer to the textbook
or to the internet.
So what you do is I can't just
magically replace this node
with one of the subtrees,
but we talked right
before this about Next Greater,
so finding a node's successor.
If this node has both a left
subtree and a right subtree,
then I know that if I
call Find Successor on it,
I'm going to go
somewhere inside here,
and I'm going to find a node
somewhere in here all the way
to the left that is
this guy's successor.
So what I'm going
to do is I'm going
to delete this node instead, and
then I'm going to take its key
and put it up here.
So if I want to delete 8, what
I do is I find its successor,
then I delete it, then I take
the 15 that was here-- you
can see it, right?
It's still there.
Put it here.
So the reason this works
is that everything here
is greater than this guy.
Everything here is
smaller than this guy.
This is the next node that's
greater than this guy,
but everything else
is bigger than it,
right, because we wanted
it to be a successor.
So if I take this value
and I put it up here,
everything in here is still
going to be greater than it.
This is a successor of this
guy, so everything here
is still going to be
smaller than the successor.
Great.
In order to do a delete,
I find the successor,
and then I call Delete on it.
How do I know that
this will end?
How do I know that
I'm not going to go
into a loop that runs forever?
AUDIENCE: Because it's not--
AUDIENCE: It's acyclic, right?
PROFESSOR: OK.
First answer, good.
Eventually, worst case, I'm
going to get to the maximum,
and then not going on have to
delete the successor anymore.
Now, another thing
to note here is
that if this guy is the
successor of this guy,
it can't have anything on the
left, because if it would,
then whatever is down here
has to be bigger than this,
and whatever's to
the left of this node
has to be smaller than this.
But we said that this is
the successor of this,
so there's nothing here.
So this will be one of the easy
cases that we talked about.
The successor
either has no kids,
or it has only one
child, only one subtree.
So then I can delete it
using one of the easy cases.
So in fact, worst case that
happens in a delete is my node
has two subtrees.
Then I find the successor that's
only going to have one subtree,
I change my links
there, and I'm done.
What is the running
time for Delete?
AUDIENCE: Is it order h,
because you should do it
all the way down to the
bottom of the tree, right?
PROFESSOR: You have
the right answer.
Let's see why it's order h.
It has to be order h, right?
Otherwise, the tree
would be too slow.
If it's order N, then it's bad.
So why would Delete be order h?
This was a heap, right,
so I can't use this.
I'm going to write
"delete" here again.
So the first thing you do is
you have to search for the key,
right?
That's order h.
Now, if it's a happy case,
if it's case one or two,
you change some links
and you're done.
What's the time for that?
AUDIENCE: Constant.
PROFESSOR: Constant.
So happy case, order h for sure.
Now sad case.
If you have two
children, what do you
have to do after you realize
that you have two subtrees?
AUDIENCE: Find the successor.
PROFESSOR: OK.
What's the running time
for finding a successor?
AUDIENCE: Order h.
PROFESSOR: Order h.
Once I find the
successor, what do I do?
Call Delete on that,
and what happens?
It's a happy case or a sad case?
AUDIENCE: It's a happy case.
PROFESSOR: Happy case,
a few links get swapped,
constant time.
So worst case, order
h plus order h.
Order h.
So insertions are order
h, deletions are order h.
AUDIENCE: The first one.
Because the second one is
from finding the successor.
What is the first one for?
PROFESSOR: Finding the
node for a key in the tree.
So if I say Delete 8,
then you have to find 8.
If I give you the node,
then you don't have that.
Good question.
It's a good question.
Thank you.
So that's insertion.
That's deletion.
Let's look at the
code for Delete.
Looks kind of long.
So lines through 21,
happy case or sad case?
Try to do it by
looking at the "if"
instead of looking
at the comments.
So lines through 21 for Delete.
AUDIENCE: On this tree?
Which tree, because
there are two deletes?
PROFESSOR: Oh really?
Sorry.
Why do we have two deletes?
AUDIENCE: There's BST Delete and
then there's BST Node Delete.
PROFESSOR: So BST Delete.
Finds the node, and then
calls Delete on the node.
And then if the node
is a tree's root,
then it updates the tree's root.
So let's look at
the nodes delete.
Oh, I see.
I think I was looking
at the wrong one.
Thank you.
My Delete was much
longer than yours.
So lines 3 through 12,
happy case or sad case?
Look at the "if" on
line 3 and tell me,
what case is it going for?
AUDIENCE: [INAUDIBLE].
PROFESSOR: If it doesn't
have a left child
or it doesn't have
a right child,
is that the happy
case or the sad case?
AUDIENCE: Happy.
PROFESSOR: Happy case.
So lines 4 through 12
handle the happy case.
Lines 14 through 16
handle the sad case.
Do lines 14 through
16 make sense?
Find the successor,
then swap the keys,
then delete that successor.
Now, lines 4 through
11 are pretty much what
we talked about here, except I
can't draw arrows on the board
and instead I have to
change left and right links.
Line 4 has to see if we're a
left child or a right child,
and then lines 5 through 7 and
9 through 11 are pretty much
copy paste, swap
left with right.
And they changed the links
like we changed them here.
Do we have any
questions on Deletes?
AUDIENCE: So if the
successor had a right child,
then all you do, you just do
the workaround thing where
you just--
PROFESSOR: Yep.
So the case that it
doesn't have two children.
As long as it doesn't
have both children,
you're in the happy case and
you can do some link swapping.
Are you guys burned out already?
Fair enough.
I left a part out.
What I left out is how
to augment a binary tree.
So binary trees by default
can answer the question,
what's the minimum node
in a tree in order h.
You go all the way to the
left, you find the minimum.
That's the minimum.
It turns out that if you make
a node a little bit fatter,
so if instead of storing, say,
23 in this node, I store 23,
and I store the fact that the
minimum in my left subtree
is 4, then it turns out that
I can answer the question
in constant time,
what's the minimum?
Oh gee, if you store
the minimum here,
of course you can retrieve
it in constant time, right?
The hard part is, how do you
handle insertions and updates
in the same time?
So the idea is that
if I have a node
and I have a function here,
say the minimum of everything,
if I have two children,
here they're 15 and 42,
and say the minimum
in this tree is 4
and the minimum in this tree is.
So if I already computed
the function for these guys,
how do I compute the
function for this?
AUDIENCE: [INAUDIBLE]
and compare it?
PROFESSOR: Yep.
Take the minimum of
these two guys, right?
There are some special cases
if you don't have a child.
If you don't have a left
child, then you're the minimum.
But you write down
those special cases,
and you can compute
this in how much time?
AUDIENCE: Order h, right?
PROFESSOR: What if I
already computed the answer
for the children?
How much time does
it take to compute
the answer for a single node?
AUDIENCE: Constant.
PROFESSOR: Constant.
OK.
AUDIENCE: For a tree, though.
PROFESSOR: For a
tree, it's order h.
Yeah.
You're getting ahead.
You're rushing me.
You're not letting me finish.
AUDIENCE: Are you saying that
we store the minimum value?
PROFESSOR: So for every--
AUDIENCE: Each node
has a field that
says what the minimum
value is in that tree.
PROFESSOR: Yep, exactly.
So for each node, what's
the minimum in the subtree.
So if I add a node here,
suppose I add three
and I had my minimums,
what changed?
This subtree changed,
this subtree changed,
this subtree changed, and
then this subtree changed.
So I have to update the
minimums here, here, here, here.
Nothing else changed.
Outside the path where
I did the Insert,
nothing changed, so I don't
have to update anything.
So what I do is after
the Insert, I go back up
and I re-compute the values.
So here, I'll have 3.
I go back up 3, 3, 3.
AUDIENCE: You could when
you're passing down, though.
When you're going
down that column,
you can just compare
it on the way down.
You don't have to
go back up, right?
PROFESSOR: Yep.
So the advantage of doing
it the way I'm saying it
is that you can have other
functions instead of minimum.
As long as you can
compute the function
inside the parent
in constant time
using the function
from the children,
it makes sense to compute the
function on the children first.
There's an obvious function
that I can't tell you
because that's on the Pset,
but when you see the next Pset,
you'll see what I mean.
So if you have a
function where you
know the result for the children
and you can compute the result
for the parent in constant time,
then after you do the Insert,
you go up on the path and
you re-compute the function.
When you delete, what do you do?
AUDIENCE: Same thing.
PROFESSOR: Same thing.
If this goes away, then this
subtree changed, and then
if there would be
something else here,
then this subtree changed,
but nothing else changed.
So whenever you do an Insert
or a Delete, all you have to do
is go back up the
path to the parent
and re-compute the function
that you're trying to compute.
And that's tree augmentation.
Does this make sense somewhat?
That's it.
So what you'll find
in lecture notes
is a harder way of doing
it that works for minimum,
but what I told you
works for everything.
So don't tell people I told you
how to do this for everything.
Sure nobody's going to know.
