The following content is
provided under a Creative
Commons license.
Your support will help
MIT OpenCourseWare
continue to offer high quality
educational resources for free.
To make a donation or to
view additional materials
from hundreds of MIT courses,
visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: So what I'm
going to do in this course
is discuss mostly ideas that
are already in the book called
"The Emotion Machine."
I'm sorry.
I used that title,
and the older book
called it "The Society of Mind."
The books are not
quite the same.
They overlap a bit in
material, but they're
sort of complementary.
I like the old one better,
because the chapters are all
one page long.
And they're moderately
independent.
So if you don't like
one, you can skip it.
The new book is much denser,
and it has a smaller number
of long chapters.
And I think it's--
over the years, I
got lots of reactions
from young people in
high school, for example.
Almost all of whom liked
"The Society of Mind,"
and found it easy to read,
and seem to understand it.
There are lots of criticisms
by older people who
maybe some of them
found it harder to put
so many fragments together.
Who knows?
But most of this class, most
of the things I'd like to say
are in those books.
So it's really
like a big seminar,
and my hope is that everyone
who comes to this class
would have a couple of questions
that they'd like to discuss.
And if I can't answer them,
maybe some others of you can.
So I like to think of
this as a super seminar,
and normally, I don't
prepare lectures.
And I just start off asking
if there are any questions.
And if there are not, I
get really pissed off.
But anyway, I'm going to
start with a series of slides.
So why do we need machines?
And partly, there are
a lot of problems.
Unlike most species
or kinds of animals,
humans have only been
around a few million years.
And they're very clever
compared to other animals,
but it's not clear how
long they will last.
And when we go, we might
take all the others with us.
So there are a whole
set of serious problems
that are arising, because
there are so many humans.
And here's just a
little list of things.
There's a better list in a book
by the Astronomer Royal Martin
Rees of England.
Anybody know the title?
AUDIENCE: "Our Final Hour."
PROFESSOR: Yes,
"Our Final Hour."
It's a slightly scary title.
And when I was a teenager,
World War II came to an end
with the dropping
of two atomic--
oh, this is getting terrible.
--two atomic bombs on Japan.
And I didn't believe
the first one was real,
because it was in Hiroshima.
So I assumed that
the US had somehow
made a big underwater tanker
with 20,000 tons of TNT,
and some few grams of
radium or something,
and blown it up in the harbor.
And first, it flew
an airplane over,
dropping some little thing.
And this was to fool the
Japanese into thinking
that we have an atomic bomb.
But when they did it,
again, over Nagasaki,
that wasn't feasible.
And when I was in grade school,
Sometimes, if I said
something very bright,
I would hear a
teacher saying, maybe
he's another J.
Robert Oppenheimer.
Because that was the
name of a scientist who
had been head of the
Manhattan Project, and he was,
I think, three or four years
earlier in grade school
than I was.
And I thought it was
very strange for a person
to have a first name just being
a letter rather than a name.
Many years later when I was at
Princeton in graduate school,
I met the Robert Oppenheimer,
and that was a great pleasure.
And in fact, he took me
to lunch with a couple
of other people I admired,
namely, Girdle and Einstein,
which was very exciting.
Except I couldn't
understand Einstein,
because I wasn't used to people
with a strong German accent.
But I understood
Girdle just fine.
And after that lunch was over,
I went and spent about a year
learning about Turing machines,
and trying to prove theorems
about them, and so forth.
So anyway, in the
course of these talks,
we'll run across a
few of these people.
And here's a big
list of the people
that I'm mostly indebted
to for the ideas
in the society of mind
and the emotion machine.
The ones in blue are
people I've actually met.
It would be nice to
have met Aristotle,
because no one really
knows much about him.
But you really should read,
just skim through some of that,
and you'll find that this
is a really smart guy.
We don't know if he
wrote this stuff,
or if it were compiled
by his students,
like a lot of Feynman's writing
is and Von Neumann's writing is
edited from notes
by their students.
Anyway, the astonishing
thing about Aristotle
is that he seems
to be slightly more
imaginative than most
cognitive scientists you'll
run into in the present day.
It would have been
nice to know Spinoza,
and Kant, and the others also.
Freud wrote 30 or 40 books.
Did he fall off this list?
There he is.
I just made this
list the other day,
and I was looking
up these people
to find their
birthdays and stuff.
Yes?
AUDIENCE: Why are there
no Eastern philosophers
in history?
PROFESSOR: Because they're
religious as far as I can see.
AUDIENCE: Religious?
PROFESSOR: Well, who would
you-- would you say, Buddha?
AUDIENCE: No, I mean, just
Eastern thinkers thought that.
PROFESSOR: Name one.
Maybe I've never heard of them.
AUDIENCE: Confucius.
PROFESSOR: Who?
AUDIENCE: Confucius.
PROFESSOR: Confucius?
AUDIENCE: Or [INAUDIBLE]
great thinkers from China.
PROFESSOR: Well, I
only know of them
through aphorisms,
single proverbs,
but I don't know that Confucius
had a theory of thinking.
You think he did?
AUDIENCE: There are a lot
differences of thoughts,
and I think they probably
do have [INAUDIBLE]..
PROFESSOR: Well, I've
looked at Buddhist theories,
and they're--
I don't think they
would get a C plus.
And one problem is that
there are cultures-- there's
something about Greek culture.
Because it had science.
It had experiments.
Somebody has a theory, and they
say, like Epimenides Lucretius.
Somewhere in the
society mind, I think
I quoted Lucretius about
translucent objects.
And he says, they have
the particular appearance,
because the rays of
light bounce many times
before they get to the surface.
So you can't tell
where they started.
And I don't find in
eastern philosophy theories
that say, here's what, I
think, and here's a reason why.
I've looked at Buddhist
stuff, and it's strange lists
of psychological principles.
Every one of which
looks pretty wrong,
and they make nice two
dimensional diagrams.
But no evidence for
any of them, so I
don't know whether
to take it seriously.
AUDIENCE: I think knowledge
is from observation.
I think you're right that
in some them probably
didn't really test it, because
a lot of the ideology cannot be
tested.
On the other hand,
there are scientists--
PROFESSOR: But what
can't be tested?
AUDIENCE: I mean, some of
the ideologies probably.
PROFESSOR: If they
can't be tested,
why should one look at it twice?
AUDIENCE: Test it in terms
of something logically.
I don't know.
Like culture, can
you test culture?
PROFESSOR: OK, I think
this is a serious argument.
It seems to me that science
began a little bit in China,
a little bit in India.
In the Arabic world, they got
up to the middle of high school
algebra, but then--
AUDIENCE: That's the foundation.
PROFESSOR: What?
AUDIENCE: That's the foundation.
PROFESSOR: Well,
but it wasn't as
good as Archimedes, who got
to the beginning of calculus.
So if you look at most
cultures, they never
got to the critical point
of getting theories, doing
experiments, discussing them,
and then throwing them out.
And so if you look at
Buddhist philosophy,
it's 2,500 years old.
If you look at Greek physics,
yes, Archimedes almost
got calculus, and he got
lots of nice principles.
And Buddha mentions,
at some point,
if you want to weigh an
elephant, put him in a boat.
And then take the elephant
out and put rocks in,
until the boat sinks
to the same level.
So there, you see a good idea.
But if you look at the
history of the culture,
if people still say, this
thousand year old stuff
is good, then you should
say, no, it's not.
AUDIENCE: By the way, same
story about the elephants.
There's like a story in Chinese
history that has the same.
PROFESSOR: Sure.
AUDIENCE: I mean, maybe there's
no one person that [INAUDIBLE]..
PROFESSOR: No, but the
question is, why did it stop?
Why did it stop?
Ancient wisdom is
generally not very good,
and we shouldn't
respect it for too long.
And that's--
AUDIENCE: [INAUDIBLE]
past where everybody's
standing on the giant's
shoulders, right?
PROFESSOR: No, we
got rid of alchemy.
We got rid of--
what do you call it?
What's caloric?
You jump off their shoulder.
You don't stay on them, so
it's good to know history.
But if the history
doesn't get anywhere,
then you don't want
to admire it too much.
Because you have to
ask, why did it stop?
What went wrong?
And usually, it went
wrong, because barbarians
came in and--
well, you know what
happened to Archimedes.
Some Roman killed him.
Anyways--
AUDIENCE: [INAUDIBLE].
I'm sorry.
PROFESSOR: No, it's
a good question.
Why didn't science happen
a million years ago?
Because humans are five
million years old, so
what took it so long?
AUDIENCE: [INAUDIBLE]
PROFESSOR: No, it's more--
AUDIENCE: [INAUDIBLE]
PROFESSOR: Sure, OK.
Do you have a theory
of why science
didn't develop for so long?
In most cultures, it
might be religion,
which is a sort of science
that doesn't use evidence,
and in fact, kills
people who try to get it.
So there are
systematic reasons why
most cultures failed, and
maybe somebody has written it.
Is there a book on why science
disappeared, except once?
It's rather remarkable.
Isn't it?
After all, the idea, if
somebody says something,
and somebody else
says, OK, let's
do an experiment to
see if that's right,
you don't have to
be very bright.
So how come it didn't happen
all the time everywhere?
Here he is.
AUDIENCE: I don't know
the answer to that,
but I know Paul Davies has
sort of an anecdote about that.
Or he's exactly
speculating, even
in Europe when it did
happen was a fluke.
And he gives the example of
suppose an asteroid or a comet
crashed in Paris in--
I forget what year he gives.
--1150, or 1200, or something.
Then what?
Whether it's science [INAUDIBLE]
as a thought problem.
PROFESSOR: History
is full of flukes.
I'm trying to remember
who wrote that nice book
about the plague, some woman.
And she mentions
that this was spread
by rats and fleas or something.
And 30% or 40% of the population
of many countries in Europe
died, and the next generation
had a lot of furniture.
The standard of
living went way up,
so anyway, here's a
list of disasters.
Oh, come on.
And Martin Rees is
the royal astronomer
and has that book about
the last hour or whatever.
I'm making another longer list.
But he has lots of
obvious disasters,
like some high school
student looks up
the genetic sequence
for smallpox virus
has been published, and now, you
can write a list of nucleotides
and send it somewhere.
And they'll make it for about
$0.50 or $1 per nucleotide.
So for a couple of
hundred dollars,
you can make a virus
or a few hundred.
So one possibility is that
some high school student
makes some smallpox only gets
it wrong, and it kills everyone.
So there are lots of
disasters like that,
and no one knows what
to do about that.
Because the DNA
synthesis machinery
is becoming less and less
expensive, and probably
the average rich private
high school could afford one.
So there are lots of other
things that could happen.
But one particular one is this
graph, which I just made up.
An interesting fact
is that since 1950
when the first antibiotics
started to appear,
as I mentioned, I was
a kid in the 1940s.
And penicillin had
just hit the stands,
and there wasn't much of it.
And there was a researcher
who lived a few blocks from us
whose dog had cancer.
So its father--
I don't know what you
call the owner of a dog.
--sneaked some
penicillin out of the lab
and gave it to the
dog, who died anyway.
But he said, well, nobody's
tried penicillin on cancer yet.
Maybe it will work.
And a lot of people
were mad at him,
because he probably cost
some human its life.
But he said, he might have
saved a billion humans
their lives, so ethics.
Ethicists are people who give
reasons not to do things,
and I'm not saying
they're wrong.
But it's a funny job.
Anyway, since that
sort of thing happened
and medicine began
to advance, people
have been living one
year longer every 12.
So it's 60 years since 1950,
so that's five of those six.
So they're living
six or seven years
longer now than they
were when I was born.
And somebody
mentioned that curve
stopped the last few
years for other reasons,
but anyway, if you
extrapolated, you'll
find that the lifespan is
going to keep increasing.
How much we don't know--
another problem is
that you might discover
enough about genetics
to get rid of most
of the serious diseases.
Maybe just 20 or 30 genes are
responsible for most deaths
right now.
And if you could fix those,
which we can't do yet,
there's no way to change
a gene in a person.
Because invading all the
cells is a pretty massive
intervention, but
we'll get around that.
And then it might be
that people suddenly
start living 200 or 300 years.
Now at some point, the
population has to slow down.
So you can only
reach equilibrium
with one child per family
and probably less than that.
So all the work has to be
done by 200 or 300-year-olds,
and let's hope they're
good and healthy.
So anyway, I think
it's very important
that we get smart robots,
because we're going
to have to stem the population.
And I hope people will live
longer and blah, blah, blah.
So these robots have to be smart
enough to replace most people,
and how do you make
something smart?
Well, artificial intelligence
is the field whose goal with
has been to make
machines that do things
that we regard as
smart, or intelligent,
or whatever you want to call it.
And the idea of seriously
making machines smart
has roots that go back
to a few pioneers,
like Leibniz, who
wrote about automata
and that sort of thing.
But the idea of a
general purpose computer
didn't appear till the 1930s
and '40s in some sense.
The first form of the
general purpose computer
appears really in
the 1920s and '30
with the work of
a mathematician,
Emil Post at NYU, who
I happen to never meet.
But we had some
friends in common,
and he had the idea
of production rules.
And basically,
rule based systems
and prove various
theorems about them.
Then Kurt Girdle showed
that, if you had something,
like a computer or
a procedure that
had the right kinds
of rules, it could
compute all sorts of things.
But there were some things
it couldn't compute,
unsolvable problems, and that
became an exciting branch
of mathematics.
And the star thinker
in that field
was Alan Turing, who
invented a very simple kind
of universal general
purpose computer.
Instead of a random
access memory,
it just had a tape, which
it could write on, and read,
and change symbols.
And it would go back and forth.
And if it's in state
x at, say, symbol y,
it will print symbol
z over the x and move
to the left or right
and just a bunch
of rules like that, where
it was enough to make
a universal computer.
So from about 1936,
it was sort of
clear to a large
mathematical community
that these were great things.
And a couple of general
purpose light computers,
very simple ones, were
built in the 1930s and more
in the 1940s.
And in the 1950s,
big companies started
to make big computers, which
were rooms full of equipment.
But as you know, most
programs could only
do some particular thing, and
none of them were very smart.
Whereas a human can handle
lots of kinds of situations.
And if you have one that
you've never seen before,
there's a good chance
you'll think of a new way
to deal with that and so forth.
So how do you make a machine
that doesn't get stuck almost
all the time?
And I like to use the
word resourcefulness.
Although, I left an
R out of that one.
Is there a shorter word?
So here's a good example.
My favorite example
of a situation
where a person is
born, more or less,
with a dozen different ways
of dealing with something.
And the problem that I imagine
that you're dealing with
is this.
My favorite example
is I'm thirsty,
so I see that glass of water.
And I do that and get it.
Actually, I am.
On the other hand,
if I were here,
I would never in a
whole lifetime do this.
You never walk out
a window by mistake.
We're incredibly reliable, so
how do I know how far it is?
And that slide shows you 12
different ways that your vision
system--
that's only your vision system.
--has to measure distances.
So gradients, if things
are sort of blurry,
then they must be
pretty far away.
That's sort of on a
foggy day outside.
Here's a situation.
If you assume those are both
chairs of the same size,
and you know that this chair
is about twice as far away
as that, although, you don't--
well, and how far
away they are pretty
much by the absolute size.
If you have two
eyes that work well,
then, if something is
less than 30 feet away,
you can make a pretty good
estimate of its distance
by focusing both
eyes on some feature.
And your brain can tell how far
apart your eyes are looking,
so there's 12 different things.
It's more than you need.
Lots of people are
missing half of those.
Lots of people have very
poor vision in one eye.
Some people cannot
fuse stereo images,
even though both eyes
have 20/20 vision.
And in some cases, nobody
knows why they can't do that.
I think I once took a
test for being a pilot,
and they wanted to be sure you
could do stereo vision, which
seem very strange.
Because if you're an
airplane, and you're
less than 30 or 40 feet away
from something, it's too--
you could use stereo.
But it's too late.
Anyway, that's interesting.
See if you can think of an
example, where a person has
even more 12 of these.
But it's pretty
amazing, isn't it?
It's more redundancy.
This is too hard to
read, but somehow, I
found it in Aristotle essay the
idea that you should represent
things in multiple ways.
You might describe a house.
One person might
describe a house
as a shelter against destruction
by wind, rain, and heat.
Another might describe it as a
construction of stones, bricks,
and timbers.
But a third possible
description would say,
it was in that form in that
material with that purpose.
So you see there's two
different descriptions.
One is the functional
description.
It's a shelter.
The second one is a structural
description, how it's made.
And Aristotle says, which
is the better description?
And he dismisses the material
one or the functional one
is not rather the
person who combines
both in a single statement.
And then I found a paragraph
by Fineman, who says,
every theoretical
physicist who is any good
knows six or seven
different ways
to represent exactly
the same physics.
And you know that
they're all equivalent,
but you keep them all
in your head hoping
that they will give you
different ideas for guessing
I should put more dots.
Anyway, that whole
argument is to say
that the interesting
thing about people
is that they have so
many ways to do things,
and perceive things,
and think of things.
And in some cases, we
even know that there
are different parts
of the brain that
are involved in one
aspect or another
of constructing those
different representations
or descriptions.
If you look at one
of my favorite books,
it weighs about 20 pounds.
It's the book on the nervous
system by Kandel and Schwartz.
And the index to that book
is quite a lot of pages long,
and it mentions 400 different
structures in the brain.
So the brain is not like the--
well, I shouldn't
make fun of the liver.
Because for all
I know, the liver
has 400 different many
processes for doing things.
But the brain has
distinguishable areas
that seem to perform several
hundred different functions.
And with a microscope,
at first, they all
look pretty much the same.
But if you look closely, you
see slightly different patterns
of how the most layers of
the cortex of the brain, most
parts of it have six layers,
and each has a population
of different kinds of cells.
There are a lot of cross
connections up and down
and sideways to other.
They're arranged in
columns of between 400
and the 1,000 cells, and
you have a couple of million
of those.
And there are lots
of differences
between the columns
in different areas,
and we know some
of the functions.
In most cases, we don't know
much about how any of them
actually work with the
main exception of vision,
where the functions of the
cells in the visual cortex
are fairly well
understood at low levels.
So we know how that
part of the brain
finds the edges and
boundaries of different areas,
and textures, and regions
of the visual field.
But we do not know
even a little bit
about how the brain
recognizes something
as a chair, and an overhead
projector, and a CRT screen,
and that sort of thing.
The kind of question that I got
interested in was, how can you
have a system, which has a very
large number of different kinds
of computers?
Each of which by itself might be
relatively simple or might not,
I suppose.
And how could you
put them together
into a larger system, which
could do things, like learn
language, and prove
theorems, and convince
people to do things
that they would never
have dreamed of doing
five minutes earlier,
and stuff like that?
Now the first sort of
things I was interested
in was, in fact, how to simulate
simple kinds of nerve cells.
Because in the 1950s, there
was about almost 100 years,
really more like 50 years of
science discovering things
about neurons and nerve cells,
the axons, and dendrites
that they use to communicate
with other neurons.
So if you go back to 1890,
you find a few anatomists
discovering some of the
functions or connections
of neurons in the brain.
And you find a few
experimental physicists.
There was no oscilloscope yet,
but there were very high gain
galvanometers, which
could detect pulses
going along a nerve fiber.
And by 1900, it was
pretty clear that part
of the activity in a nerve
cell was chemical and part
was electrical.
And by 1920 or '30 with
the cathode ray tube
appearing mostly
because of television,
but it became possible to
do a lot of neurophysiology
by sticking needles in brains.
The vacuum tube
appears around 1900,
and you can make amplifiers
that can see millivolts
and then microvolts.
So in the beginning
of the 20th century,
there was lots of progress.
By 1950, we knew a lot
about the nervous system,
but we still don't
know much about how you
learn something in the brain.
It's quite clear that the things
called synapses are involved.
The connections
between two neurons
become better at
conducting nerve impulses
under some
conditions, but no one
knows how higher level knowledge
is represented in the brain
yet.
And the Society of Mind book had
a lot of theories about that.
And in particular,
there was a theory
called k line's, knowledge
lines, or something
that came partly
from me and partly
from a couple of other
researchers named
David Waltz and Jordan Pollock.
That's a sort of nice theory
of how neural networks might
remember higher level concepts.
And for some reason,
although that kind of work
is from around 1980,
which is 30 years ago,
it has not hit the
neuroscience community.
So if you look at the emotion
machine book or the society
minded in Amazon, you
might run across a review
by a neurologist named
Richard Restak, who
says that Minsky makes up a
lot of concepts, like K-lines,
and micronemes, and stuff
like that, that nobody's ever
heard of.
And there's no
evidence for them,
and he ignores the
possibility that it
isn't the nerve cells in the
brain that are important.
But the supporting
tissues called
glia, which hold the
neurons up and feed them.
And he goes on for a couple
of insane paragraphs.
It's very interesting,
because it doesn't
occur to him that you
can't look for something,
until you have the idea of it.
So here is this 30-year-old
idea of K-lines, and go and ask
your favorite neurologist,
neuroscientist what it is.
And he said, oh, I think
that's some AI thing,
but where's the evidence for it?
What do you suppose is
my reaction to that?
Who is supposed to
get the evidence?
So it seems to me that there's
a strange field in neuroscience,
which is that it
doesn't want new ideas,
unless you've proved them.
So I try to have
conversations with them,
but get somewhat tired of it.
Anyway, in this course, I'm
taking the opposite approach,
which is that we don't
want a theory of thinking.
We want a lot of them,
because probably, psychology
is not like physics.
What's the most wonderful
thing about physics?
The most wonderful thing is
that they have unified theories.
There wasn't much of a
unified theory, until Newton,
and he got these
three wonderful laws.
One was the gravitational idea
that bodies attract each other
with a force that's
the inverse square
of the distance between them.
Another is that kinetic
energy is conserved.
I forget with the third one is.
Oh, equal reaction is
equal and opposite.
If two things collide,
they transfer equal amount
of momentum to both.
There was a little problem
up to Newton's time.
Galileo got some of those
ideas, and my impression
from reading him is that he
has a dim idea that there
are two things around.
There's kinetic
energy, which is MV--
oops, momentum is MV.
And there's kinetic energy,
which is MV squared,
and he doesn't have the
clear idea that there
are two different things here.
And you can't blame him.
I would think-- you wouldn't
think that two quantities would
combine in two different
ways to make two
important different concepts.
Well, that got clear
to Newton somehow,
and Galileo is a bit muddled.
Although, he gets almost all
the consequences of those things
right, but he doesn't get the
orbits and things to come out.
Anyway, what's happened in
artificial intelligence,
like most fields, is
that people said, well,
let's try to understand
thinking and psychology.
And let's use physics as
our model, so what we want
is to get a very small
number of universal laws.
And a lot of psychologists
struggled around to do that,
and then they
gradually separated.
So that there were
some psychologists,
like Bill Estes, who
worked out some very nice
mathematical rules for
reinforcement based learning,
got a simple rule.
If you designed an
experiment right,
it predicted pretty
well how many trials
it would take a rat,
or a pigeon, or a dog,
or whatever to learn a certain
thing from trial and error.
And Este's got a set of
four or five rules, which
looked like Newton's laws.
And if you designed your
experiment very carefully
and shielded the animal from
noise and everything else,
which is what a physicist would
do for a physics experiment,
the reinforcement theories
got some pretty good models
of how to make a machine learn.
But they weren't good enough.
So here's a whole
list of things that
happened in the early years
of cognitive psychology
when people were trying to
make theories of thinking,
and they were imitating
the physicists.
By physics envy to
borrow a term of Freud,
the idea is, can you
find a few simple rules
that will apply to
very broad classes
of psychological phenomena?
And this led to various
kinds of projects.
Lots of neural network,
and reinforcement,
and statistical
based methods led
to learning machines that
were pretty good at learning
in some kinds of situations.
And they're becoming very
popular, but I don't like them.
Because, if you have a lot
of variables, like 50 or 100,
then to use a
probabilistic analysis,
you have to think
of all combinations
of those variables.
Because if two of them
are combined in something,
like a exclusive or
a manner, you know,
I just put the light
pen in a pocket.
It's either in the left
pocket or a right pocket.
It can't be both.
That's an x or.
That will cause a lot of
trouble to a learning machine.
And if there are a
hundreds variables,
there's no way you
could decide which
of the two to the 100th Boolean
combinations of those variables
you should think about.
So lots of statistical
learning systems
are good for lots
of applications.
But they just won't cut it
to solve hard problems, where
the hypothesis is a
little bit complicated
and has seven or eight variables
with complicated interactions.
Most statistical
learning people assume
that, if you get a
lot of partial ones,
then you can look for
combinations of ones
that have high correlations
with the result. Then
you can start combining
them, and things
get better and better.
However, mathematically, if
an effect you're looking for
depends on the exclusive
or of several variables,
there's no way to approach that
by successive approximations.
If any one of the
variables is missing,
there won't be any
correlation of the phenomenon
with the others.
Anyway, that's a long
story, but I think
it's worth complaining about.
Because almost all
young people who
start working on
artificial intelligence
look around and
say, what's popular?
Statistical learning,
so I'll do that.
That's exactly the way to
kill yourself scientifically.
You don't want to get
the most popular thing.
You want to say, what are
my really good at that's
different?
And what are the chances that
would provide another thing?
End of long speech.
Another problem in
the last 30 years--
and as you'll see
during my lectures,
I think a lot of
wonderful things
happened between 1950
when the idea of AI
first got articulated
in the 1950s.
And then the 20 years after
that from 1960 to 1980,
a lot of early experiments--
and I'll show you some of them.
--looked very promising.
In fact, they may
be-- here we go.
1961, Jim Slagle was a young
graduate student here at MIT.
He was blind.
He had gotten some
retinal degeneration thing
in his first or second
year of high school.
He was told that he would
lose all his vision,
and there was no
treatment or hope.
So he learned Braille
while he could still see.
And when he got to MIT,
he was completely blind,
but there was a nice big parking
lot in technology square.
And he would ride a bicycle.
And people, like Sussman,
and Winston, and whoever
was around, would yell
at him, telling him where
the next obstacle would be.
Jim got better and
better at that,
and nothing would stop him.
And he decided he would
write a program that-- oh, I
wrote a program that
would take any formula
and find its derivative.
It was really
easy, because there
were just about five rules.
Like if there's a
product UV, then
you compute U times the
derivative of V plus V
times U of DV plus DVU.
So I wrote a 20
line list program
that did all the
algebraic expressions,
and what it would do is
put Ds in the right place.
And then it would go back
through the expression again.
Wherever it saw a D, it would
do the derivative of the thing
after that and nothing to it.
So Slagle said, well,
I'll do integrals.
And we all said, well,
that's very hard.
Nobody knows how to do it.
And in fact, in Providence
at the home of the American
Mathematical Society,
there is a big library
called the Bateman
Manuscript Project,
which has been collecting all
known integrals for 100 years.
And when anybody
finds a new integral
that they can integrate
in closed form,
they send the formulas to the
Bateman Manuscript Project,
and some hackers there
develop ways to index it.
So if you had an
integral, and you
didn't know how to integrate
it, you could look it up.
And that was pretty big.
I should say that
Slagle succeeded
in writing a
program that managed
to do all of the
kinds of integrals
that one usually found on the
first year calculus course
at MIT and got an A in those.
He couldn't do word problems.
And the uncanny
thing is that, if it
was a problem that usually
took a MIT student five
or 10 minutes, Slagle's program
would take five or 10 minutes.
It's running on an IBM 701 with
a 20 millisecond cycle time.
It's incredibly slow.
You can type almost that fast
and 16 K of words of memory.
So there's no
significance whatever
to this accident of time.
It would now take a
microsecond or so.
It would be 1,000 million
times faster than a student.
Quite remarkable.
I don't have a slide.
Joel Moses, then Slagle
went and graduated.
Joel Moses was another
student, who is--
is he provost now or what?
He got tired of it.
A terrific student,
and he set up
a project called maxima for
project Max Symbolic Algebra
and got several people all
over the country working
on integration.
And at some point, a couple
of them, Bobby Caviness and--
forget the other one--
found a procedure that could,
in fact, integrate everything--
every algebraic
expression that has a--
can be integrated
in closed form.
I forget the couple
of constraints on it.
And that became a
widely used system.
It ultimately got replaced by
Steven Wolfram's Mathematica.
But Maxima was sort of
the world-class symbolic
mathematician for
quite a few years.
And Moses mentioned to me he had
read Slagle's program thesis.
And it took him a couple
of weeks to understand
the two pages of--
or three pages of--
Lisp that Slagle had written.
Because being blind,
Slagle had tried
to get the thing into as
compact a form as possible.
But that's symbolic.
It's too easy.
It was in a more ambitious one,
which was, three years later,
Dan Bobrow, who is now a vice
president doing something
at Xerox--
and it solved
problems like this.
The gas consumption of my
car is 15 miles per gallon.
The distance between Boston
and New York is 250 miles.
What is the number
of gallons used
on a trip between
Boston and New York?
And it chomps away
and solves that.
It has about 100 rules.
It doesn't really know what
any of those words mean.
But it thinks that the
word "is" is equals.
The distance
between-- doesn't care
what Boston and New York is.
It has a format thing which
says the distance between two
things.
And it never bothers to--
you see, because the phrase
"Boston and New York"
occurs twice in the
example, it just
replaces that by some symbol.
It was fairly remarkable.
And generally, if you
had an algebra problem,
and you told it to Bobrow,
Bobrow could type something in,
and it would solve it.
If you typed it in,
it probably wouldn't.
But it was-- it had more
than half a chance, or less--
about half a chance.
So it was pretty good.
And if you look at an
out-of-print book I wrote
called--
I compiled, called--
Semantic Information Processing,
most of Bobrow's program
is in that.
So that's 1964.
I'll skip Winograd,
which is, perhaps,
the most interesting program.
This was a program where you
could talk to a robot that--
I don't have a good
picture in this slide.
But they're a bunch of
blocks of different colors.
They're all cubes in the--
or rectangular blocks.
And you can say, which
is the largest block
on top of the big blue block?
And it would answer you.
And you could say, put
the large red block
on top of the small green
block, and it would do that.
And Winograd's program was,
of course, a symbolic one.
We actually built a robot.
And I guess we built it second.
Our friends at
Stanford built a robot.
And they imported
Winograd's program.
And they had the robot actually
performing these operations
that you told it
to do by typing.
And it was pretty exciting.
My favorite program
in that period
was this one, because
it's so psychological.
This is called a
geometrical analogy test.
And it's on some IQ tests.
A is to B as C is to which
of the following five?
And Evans wrote a set of rules
which were pretty good at this.
Did as well as 16-year-olds.
And it picks this one.
And if you ask it why,
it says something like,
I don't have a reason.
It moves the largest object
down or something like that,
makes up different reasons.
So you see, in some sense,
we're going backwards in age.
Because we're going from
calculus, to algebra,
to simple analogies.
Oh, there it is.
That's one where the
largest object moves down.
I don't know why I
have two of them.
These are for another lecture.
OK.
So that was a period in
which we picked problems
that people considered hard,
because they were mathematical.
But when you think
about it more,
you see, well, those math
things are just procedures.
And it's once you know
what Laplace, and Gauss,
and those mathematicians--
Newton and people--
did, you can write down
systematic procedures
for integrating,
or for solving simultaneous
algebraic constraint
equations, or things like that.
And so there's
very little to it.
So in some sense,
if you look at the--
what you're doing in
math in high school,
in education, you're
going from hard to easy.
It's just that people aren't--
most people aren't very good
at obeying really simple rules,
because it's so hideously
boring or something.
So we gradually
started to ask, well,
why can't we make
machines understand
everyday things, and the
things that everyone regards
as common sense, and
people can do so you
don't need machines to do them?
And one of my favorite examples
is, why can you pull something
with a string but not push?
And there's been a lot of
publicity recently about
that interesting program
written at a group at IBM
called Watson, which is
good at finding facts
about sportspeople, and
celebrities, and politics,
and so forth.
But there's no way it could
understand why you could push--
pull something with a
string but not push.
And I don't know of any
program that has that concept
or way of dealing with it.
So that's what I
got interested in.
And starting around the--
maybe the middle 1970s or
late 1970s, several of us
started to stop
doing the easy stuff
and try to make
theories of how you
would do the kinds
of things that people
are uniquely good at.
I don't know if animals--
well, I don't know.
I'm sure a monkey wouldn't try
to push anything with a string.
Maybe it does it very
quickly, and you don't notice.
And one aspect of
commonsense thinking
is going right back to
that idea of vision having
a dozen different
systems, is that, whatever
a person normally is doing, they
are probably representing it
in several different ways.
And here's an actual scene of
two kids named Julie and Henry
who are playing with blocks.
It's pretty hard to
see those blocks.
And you can think that Julie
is thinking seven thoughts.
I'd like to see a longer list.
Maybe a good essay would
be to take a few examples
and say, what are the
most common micro-worlds?
See physical, social, emotional,
mental, instrumental--
whatever that is--
visual, tactile, spatial.
She's thinking all these things.
What if I pulled out
that bottom block?
You can't see the
tower very well.
Should I help him or
knock is tower down?
How would he react?
I forgot where I left
the arch-shaped block.
That was real.
It's somewhere over here.
But I don't think we could--
maybe it's that.
I don't know.
I remember, when
it happened, she
mentioned that she
reached around,
and it wasn't where
she thought it was.
So commonsense thinking
involves this--
in most cases, I think,
several representations.
I don't know if it's as many
as seven, or maybe 20, or what.
But that's the kind of thing
we want to know how to do.
OK, I think I'll stop,
and we'll discuss things.
But in the next lecture,
I'll talk about a model
of how I think thinking works.
What's the difference
between us and our ancestors?
We know we have a larger brain.
But if you think about it,
if you took the brain that
you already had and say--
trying to remember the name
of the little monkey that
looks like a squirrel,
jumps around in trees.
Anybody know what--
AUDIENCE: Spider monkey?
PROFESSOR: What?
AUDIENCE: Spider monkey?
PROFESSOR: Maybe.
It's a squirrel-like thing.
You wouldn't know
it was a monkey
till you took a close look.
AUDIENCE: Lemur, maybe?
PROFESSOR: Maybe.
Lemur?
I don't-- I forget.
I'll have to-- anyway, if you
just made the brain bigger,
then the poor animal would
be slower, and heavier,
and would need more food,
and take longer to reproduce.
The joke about difficulty
to give birth--
I don't know if any animal has
the problem that humans have.
A lot of people die and so on.
So how did we evolve new
ways to think and so forth?
And my first book,
The Society of Mind,
had this theory that
maybe we evolved
in a series of higher
and higher levels,
or management structures,
built on the earlier ones.
And this particular
picture suggests
that I got this idea from
Sigmund Freud's early theories.
There's been a lot of
Freud bashing recently.
You can look on the web.
I forget the authors.
But there are a couple of
books saying that he made up
all his data, and
there's no evidence
that he ever cured
anyone, and that he
lied about all the data
mentioned in his 30 or 40
books, and so forth.
AUDIENCE: Also [INAUDIBLE].
PROFESSOR: Yes, right.
But the funny part
is that if you
look at his first major book--
1895-- called The
Interpretation of Dreams,
it sort of outlines his
theory that most of thinking
is unconscious,
and it's processes
you can't get access to.
And it has a little
bit about sex,
but that's not a major feature.
And it's just full
of great ideas
that the cognitive
psychologists finally
began to get in the 1960s again
and never give credit to Freud.
So he may well have
made up his data.
But if you have a
very good theory
and nobody will listen
to you, what can you do?
His friend Rudolf
Fliess listened to him.
And there was
another paper on how
the neurons might be involved
in thinking, which was also
written around 1895, but never
got published till 1950 by--
forget who-- called "Project
for a scientific psychology."
And it's full of ideas that,
if they had been published,
might have changed everything.
Because-- anyway,
what's on your mind?
Who has-- what would
you like to hear about?
Or who has another theory?
AUDIENCE: I've got a question.
PROFESSOR: Great.
AUDIENCE: So earlier,
you talked a little bit
about how we don't really
see the neuroscience, all
of these things like
K-lines, et cetera.
Do you think it's
because they're just
really hard to find, or no-one's
actually looking for them?
PROFESSOR: Well,
Restak's review says,
he uses vague, ill-defined terms
like K-line, and microneme,
and a couple of others,
and frame, and so forth.
They're very well-defined.
They're defined better--
I mean, when he talks
about neurotransmitters,
it's as though he
thinks that chemical
has some real significance.
Any chemical would
have the same function
as any other one
provided there's
another receptor
that causes something
to happen in the cell membrane.
So you don't want to regard
acetylcholine or epinephrine
as having a mental significance.
It's just a-- it's just another
pulse, but very low-resolution.
And yes, a neurochemical
might affect all the neurons
a little bit, and raise the
average amount of activity
of some big population of cells,
and reduce the average activity
of some others.
But that's nothing
like thinking.
That's like saying, in order
to understand how a car works--
what's the most insulting
thing I could say?
Or to understand how
a computer works,
you have to understand the
arsenic, and phosphorus,
and/or--
what's the other one?
You have to understand
these atoms that are--
what?
AUDIENCE: Germanium.
PROFESSOR: Yeah, well,
that's the matrix.
So there are these one part
in a million impurities.
And that's what's
important about a computer,
isn't it, the fact that
the transistor has gain
and so forth.
Well, no, the trouble with the
computer is the transistors.
That's why practically every
transistor in a computer
is mated to another
one in opposite phase
to form a flip-flop
whose properties are
exactly the same, except
one in a quadrillion times.
In other words, everything
chemical about a computer
is irrelevant.
And I suspect that
almost everything
chemical about the brain is
unimportant except that it
causes--
it helps to make the
columns in the cortex, which
are complicated arrangements
of several hundred cells,
work reliably.
Whereas the
neuroscientist is looking
for the secret in the sodium.
When a neuron fires, the
important thing is that that
lets the sodium in and the
potassium out or vise-versa--
I forget which--
at 500 millivolts--
really quite a colossal event.
But it has no significant.
It's only when it's attached
to a flip-flop, or to something
like a K-line,
which has an encoder
and decoder of a digital sort
every few microns of its length
that you get
something functional.
So the trouble is, the poor
neuroscientists started out
with too much knowledge
about the wrong thing.
The chemistry of
the neuron firing
is very interesting, and
complicated, and cute.
And in the case of
the electric eel,
you know what happened there.
The neuron synapse, it got
rid of the next neuron.
And it just-- in
the electric eel,
you have a bunch of
synapses, or motor end plates
they're called, in series.
So instead of half a volt,
if you have 300 of those,
you get 150 volts.
I think the electric shock
that a electric eel can
give you is about 300 volts.
And this can cause
you to drown promptly
if you are in the
wrong wave when
it happens to bump into you.
I don't know why I'm
rambling this way.
You're welcome to
study neuroscience.
But please try to help them
instead of learn from them.
[LAUGHTER]
Yeah?
They just don't know
what a K-line is.
And that's a paper
that's been widely read.
It's published in 1980, and
Restak says ill-defined.
And I guess he
couldn't understand it.
Yep?
Yeah?
AUDIENCE: Why is there
no trying to make
the neuroscientists trying to
find this in the human mind?
Why don't we just, as computer
scientists, program the K-lines
and try to [INAUDIBLE]?
This is the human mind,
and we can reproduce it.
Why is not-- is that not
widespread into the computer
scientist field?
PROFESSOR: Well, there are--
I'm surprised how
little has been done.
There's-- Mike Travers
has a thesis, Tony Hearn.
There are three master's
theses on K-lines.
They sort of got them to work
to solve some simple problems.
But I'd go further.
I've never met a
neuroscientist who
knows the pioneering
work of Newell and Simon
in the late 1950s.
So there's something
wrong with that community.
They're just ignorant.
They're proud of it.
Oh, well.
I spent some time learning
neuroscience when I was--
I once had a great
stroke of luck.
When I was a--
I guess I was a
junior at Harvard.
And there was a great
new biology building
that was just constructed.
You probably know,
it's a great, big thing
with two rhinoceroses.
What are those-- what are
those two huge animals?
So this building was just
finished and half occupied,
because it was
made with a future.
So I wandered over
there, and I met
a professor named John Welsh.
And I said, I'd like
to learn neurology.
And he said, great, well,
I have an extra lab.
Why don't you-- why don't
you study the crayfish claw?
I said, great.
So he gave me this
lab, which had
four rooms, and a dark room,
and a lot of equipment,
and nobody there.
And he had worked on crayfish.
So there was somebody who went,
every week, up to Walden Pond
or somewhere, and caught
crayfish, and bring them back.
And I was a radio amateur
hacker at the time.
So I was good at electronics.
So I got my crayfish.
And Welsh showed me how to--
the great thing about
this preparation
is you can take the
crayfish, and if you--
claw-- and if you hold it
just right, it goes, snap.
It comes off.
Grows another one--
takes a couple of years.
And then there's this
white thing hanging out,
which is the nerve.
And it turns out it's
six nerves, one big one
and a few little ones.
And if you keep it in Ringer's
solution, whatever that is,
it can live for several days.
So I got a lot of switches, and
little inductors, and things,
and made a gadget,
and mounted this thing
with six wires going
to these nerves.
And then I programmed it
to reach down and pick up
a pencil like that
and wave it around.
Well, that's obviously
completely trivial.
And all the neuroscientists came
around, and gasped, and said,
that's incredible.
How did you do that?
[LAUGHTER]
They had never thought of
putting the thing back together
and making it work.
Anyway, it was--
I'm always reminding myself
that I'm the luckiest person
in the world.
Because every time I
wanted to do something,
I just happened to
find the right person.
And they'd give me a lab.
I got an idea for a microscope.
And it was this great
professor, Purcell,
who got the Nobel
Prize after a while.
And he said, that sounds
like it would work.
Why don't you take this lab?
It was in the Jefferson.
Anyway-- yeah?
AUDIENCE: I think
part of the reason
that you don't see experimental
neuroscience on things
like K-lines is that
neurons are long and thin.
So if you want to do an
experiment to actually measure
a real neural network, you
have to trace structures
with, roughly, maybe tens
of nanometer resolution.
But you need to
trace them over what
might be a couple, or even tens,
of millimeters to start to--
and you need to do this
for thousands and thousands
of neurons before you could
get to the point of seeing
something like a K-line
and understanding it.
So it's just a massive data
acquisition and processing
problem.
PROFESSOR: Oh, but
they're doing that.
AUDIENCE: They're
starting to try to.
PROFESSOR: But they don't have--
they don't know
what to look for.
Maybe you don't
have to do so much.
Maybe you just have to do a
few sections here and there
and say, well, look, there
were 400 of these here.
Now there's only 200.
It looks like this
is the same kind.
Maybe you don't have
to do the whole brain.
AUDIENCE: No, but I mean even
getting a single neuron is big.
Because it might get down to--
you need to be looking at
electron micrographs of brains
that are sliced at about
30-millimeter-- sorry, excuse
me, 30-nanometer slices.
So even just having a
single person reconstruct
a single neuron takes--
might take weeks.
PROFESSOR: Well, I don't know.
Maybe a bundle of K-lines
is half a millimeter thick.
AUDIENCE: Oh, so If you actually
do some larger-scale structure
to start off looking at, yeah.
PROFESSOR: Why not?
I just think they have
no idea what to look for.
I could give you 20 of
those in five minutes,
but nobody's listening.
AUDIENCE: So I guess
you need to know
what it looks like before you
can look for it at that scale.
PROFESSOR: What scale?
AUDIENCE: I don't know.
I mean, they know what
neurons look like.
So you know--
PROFESSOR: Yeah
AUDIENCE: You know what
to look for if you're
saying a neural net level.
PROFESSOR: I'm saying
you may only have
to look at the white matter.
AUDIENCE: Oh, all right.
PROFESSOR: Ignore the neurons.
Because the point of K-lines
is, where do these go?
And what goes into them and out?
I don't know.
It's just this idea,
let's map the whole brain,
100 billion things.
And then people like
Restak says, oh,
and there's 1,000 supporting
cells for each neuron.
He's just glorying in
the obscurity of it
rather than trying to
contribute something.
Anyway, if you run into
him, give him my regards.
[LAUGHTER]
I really wonder how somebody
can write something like that.
Yes?
AUDIENCE: Excuse my ignorance,
but what is a K-line?
PROFESSOR: The idea is that--
suppose one part of the
brain is doing something,
and it's in some particular
state that's very important,
like--
I don't know, that--
like I've just seen
a glass of water.
Then another part
of the brain would
like to know there's a glass
of water in the environment.
And I've been looking for one.
So I should try to take over
and do something about that.
Now at the moment,
there is no theory
of what happens
in different parts
of the brain for a simple thing
like that to happen, no theory
at all, except they use
the word "association."
Or they talk about, what
are the purposeful neurons?
Goal-- forget.
OK, so my theory
is that there are
a bunch of things which are
massive collections of nerve
fibers, maybe a few
hundred or a few thousand.
And when the visual
system sees an apple,
it turns on 50 of those wires.
And when it sees a pear, it
turns on a different 100 or 50
of those wires.
But about 20 of them
are the same, so forth.
In other words, it's like
the edge of a punched card.
Have you ever seen a
card-based retrieval system?
If you have a book that has--
suppose it's about physics,
and biology, and Sumatra.
And a typical 5 by 8 card
has 80 holes in the top edge.
So what do you do
is, if it's Sumatra,
you punch eight of these holes
at random, particular set.
They're assigned to Sumatra.
And then if it's--
I forget what my first
two examples were.
But you punch eight or 10
holes for each of the other two
words.
So now there are 24 punches.
Only probably four or five
of them are duplicates.
So you're punching
about 20 holes.
And now, if something is looking
for the cards that have--
were punched for
those three things,
even if there are 30 or 40
other holes punched in the card,
you stick your 20 wires through
the whole deck and lift it up.
And only cards fall out that had
those three categories punched
for.
So you see, even though
you had 80 holes,
you could punch combinations
of up to a million
different categories into that.
And if you have to put a
bunch of wires through,
you'll get all of the ones that
were punched for those cate--
the categories
you're looking for.
And you might get three
or four other cards
that will come down also.
Because all of the eight holes
were punched for some category
by accident.
Do you get the picture?
I'll send you a reference.
It was invented by a--
in the 19-- early 1940s
by a Cambridge scientist
here named Calvin
Mooers and was widely
used in libraries for
information retrieval
until computers came along.
But anyway, that's
the sort of thing
that you could
look for in a brain
if you had the concept in
your head of Zastocoding.
But I've never met
a neuroscientist
who ever heard of such a thing.
So you have this
whole community which
doesn't have a set of very
clear ideas about different ways
that knowledge or
symbols could be
represented in neural activity.
So good luck to them when
they get their big map.
They'll still have to say,
what do I do with 100 billion
of these interconnections?
Yeah?
AUDIENCE: What are your thoughts
about the current artificial
intelligence research at MIT,
such as Winston's genesis
project?
PROFESSOR: I think
Winston is just about one
of the best ones
in the whole world.
I don't know any
other projects that
are trying to do
things on that higher
level of commonsense knowledge.
He's just lost a lot of funding.
So one problem is, how do you
support a project like that?
Have you followed it?
I don't know if there's a recent
summary of what they're doing.
AUDIENCE: [INAUDIBLE]
PROFESSOR: We used
to write a big--
a new book every year
called the progress report.
The nice thing is
that we never wrote--
we had a very good form of
support from ARPA, or DARPA,
which was, every year, we'd--
every year, we'd tell
them what we had done.
They didn't-- they didn't want
to hear what we wanted to do.
And things have
turned the opposite.
So what would happen is,
every year, we'd say,
we did these great things,
and we might do some more.
Went on for about 20 years.
And it was-- and
then it fell apart.
One thing-- it's a nice story--
there was a great liberal
senator, Mike Mansfield.
And unfortunately,
he got the idea
that the defense
department was getting
too big and influential.
So he got Congress to pass a law
that ARPA shouldn't be allowed
to support anything
that didn't have
direct military application.
And Congress went for this.
And all of a sudden,
a lot of research
disappeared, basic research.
It didn't bother us much.
Because we made up
applications and said,
well, this will make
a military robot
that will go out and
do something bad.
I don't remember ever
writing anything at all.
Because-- but
anyway, around 1980,
the funding for
that sort of thing
just dried up because of
this political accident.
It was just an
accident that ARPA,
mainly through the
Office of Naval Research,
was funding basic research.
And that was a bit of history.
If you look back at
the year 1900 or so,
you see people like Einstein
making these nice theories.
But Einstein wasn't a very
abstract mathematician.
So he had a mathematician
named Herman Weyl polishing
his tensors and things for him.
And Herman Weyl's son, Joe, was
at the Office of Naval Research
in my early time.
And that office had spent
a lot of secret money
getting scientists out of
Europe while Hitler was marching
around and sending them
to places like Princeton
and other forms
of heaven in the--
in Cambridge.
And again, one of the reasons
I was lucky is that I was here.
And all these-- you know, if
you had a mathematical question,
you could find the best
mathematician in the world
down the block somewhere.
And Joe Weyl was partly
responsible for that.
And the ONR was piping all
that money to us for work
on early AI.
So it was a very
sad thing of the--
maybe the most influential
liberal in the US government
actually ruined
everything by accident.
ARPA changed its name to DARPA.
It was Advanced Research
Projects Agency.
And it had to call itself
Defense Advanced Research
Projects Agency.
Yeah?
AUDIENCE: My question
is, do you think
the achievement of artificial
intelligence is inevitable?
Or is there an obstacle
that we're just never going
to be able to overcome?
PROFESSOR: Well, Christianity
wiped out science.
That might happen tomorrow.
Only choose your religion.
AUDIENCE: Even
favorable circumstances.
PROFESSOR: It's a hard problem.
The number of people working
on advanced ideas in AI
has gotten smaller
and smaller as the--
f right now, the--
around 1980, rule-based
systems became popular.
And there were lots
of things to do.
Right now, statistical-based
inference systems
are becoming popular.
And as I said, these things
are tremendously useful.
But the problem is, if you
have a statistical system,
the important part
is guessing what
are the plausible hypotheses
and then making up the--
then finding out how
many instances of that
are correlated
with such and such.
So it's a nice idea.
But the hard problem is the
abstract, symbolic problem of,
what sets of variables are
worth considering at all when
there are a lot of them?
So to me, the most
exciting projects
are the kind that Winston
is developing for reasoning
about real-life situations.
And the one that
Henry Lieberman--
would you stand up, Henry?
Lieberman runs a
world-class group
that's working on
commonsense knowledge
and informal reasoning.
And it seems to me that that's
the critical thing that all
the other systems will need.
In the meantime,
there are people
working on logical
inference, which
has the same problem that
statistical inference has--
namely, how do you guess which
combinations of variables
are worth thinking about?
Then it seems to me that the
statistics isn't so important.
In fact, there's
a great researcher
named Douglas Lenat
in Austin, Texas
who once made an
interesting AI system that
was good at making predictions
and guessing explanations
for things.
And it was sort of like
a probabilistic system.
It had a lot of hypotheses.
And every time one of them was
useful in solving a problem,
it moved it up one on the list.
So Lenat's thing never
used any numbers.
It didn't say, this is
successful 0.73 of the time,
and now it's successful
0.7364825 of the time.
What it would do is, if
something was useful,
it would move it up
past another hypothesis.
Every now and then, it
would put a new one in.
Well, if you're
doing-- if you're
trying to solve the problem,
what do you need to know?
You want to know,
what's the most use--
what's the most likely to
be useful one, and try that.
You don't care how likely it
is to be useful as long as it's
the most, right?
I mean, if it's one in a
million, maybe you should say,
I'm getting out of here.
I don't-- I shouldn't be
working in this field at all,
or get a better problem.
But Lenat's thing did
rather wonderfully
at making theories
by just changing
the ranking of the
hypotheses that
was considered-- no numbers.
It did something very cute.
He gave it examples
of arithmetic.
And it actually-- it was
a rather long effort.
And it actually learned
to do some arithmetic.
And it invented the
idea of division
and the idea of prime number,
which was some number that
wasn't divisible by anything.
It decided that 9 was a prime--
didn't do much harm.
And it crept along.
And it got better and better.
And it invented modular
arithmetic by accident
at some point.
And it's a PhD thesis.
A lot of people didn't
believe this PhD thesis,
because Lenat lost
the program tape.
[LAUGHTER]
So he was under some
cloud of suspicion
for people thinking he
might have faked it.
But who cares?
Anyway, I think
there's a lesson there,
which is that, let's start
with something that works.
And then, if it's
really good, then
hire a mathematician who
might be able to optimize it
a little.
But the important
thing was the order.
And a good statistical one
might waste a lot of time.
Because here's this
one that's 0.78.
And here's this one that's 0.56.
And it's the next one down.
And you get a lot of experience.
And it goes up to 0.57 and 0.58.
And it never-- you know,
might be a long time
before it gets
past the other one,
because you're doing arithmetic.
Whereas in Lenat's, it would
just pop up past the other one.
Then it would get
tried right away.
And if it were no good, it
would get knocked down again.
Who cares?
So it's a real question of--
I don't know.
Mathematics is great.
And I love it.
And a lot of you do.
But there should be a name for
when it's actually slowing you
down and wasting your
time because there's
a better way that's not formal.
Yeah?
AUDIENCE: Isn't
there a saying there
are people who know
the price of everything
and the value of nothing.
PROFESSOR: That's very nice.
Yeah?
AUDIENCE: I know
you're also a musician.
So I have a
music-related question.
What do you think is
the role of music?
Like, why do all
cultures have it?
PROFESSOR: I have
a paper about it.
Oh, OK.
I've been trying to
revise it, actually.
But it's a strange question,
because there is music
everywhere.
On the other hand, I
have several friends
who are amusical.
And so I have this
theory that music
is a way of teaching
you to represent things
in an orderly fashion
and stuff like that.
Well, I have three of my
colleagues who aren't musical,
but they dance.
So they may-- it may be that--
I don't know the answer.
It's interesting.
The theory-- the first
theory in my paper
is that when you have a lot of
complicated things happening,
then the only way to learn is
to represent things that happen,
and then look at the
differences between things
that are similar, and
then try to explain
the differences, right?
I mean, what else is there?
Maybe there's something else.
So in order to become
intelligent and understand
things, you have to be
able to compare things.
And to me, the most important
feature of what's called music
is that it's divided
into measures--
bah, bah, bah, bah, bah, bah,
bah, bah, bah, bup, bah, bah.
And measures are the same number
of beats, or whatever they are.
And so now you can say da, da,
da, da, da, da, da, da, da, da,
da, da, da.
What's the different?
You changed the eighth notes in
the second one, the last four
eighth notes--
no, the two before
last-- to a quarter note.
So you're taking things that
were in different times,
and you're superimposing
those times.
And now you can
see the difference.
And the reason you
can see the difference
is that you have
things called measures.
And the measures have
things called beats.
And so things get locked
into very good frames.
Now there's some
Indian music which
has 14 measures for a phrase.
And some of the measures
go seven and five.
And I can make no sense
of that stuff whatever.
And I've tried fairly
hard, but not very.
So I don't understand
how Indians can think.
Any of you can
handle Indian music?
AUDIENCE: I guess, just to add
on what you said about this,
my favorite quote
from your paper
on music, mind, and meaning
is the one about what good is
music, about how kids play with
blocks to learn about space,
and people play with
music to learn about time.
And I think, in that
sense, both music and dance
are different ways that people
can arrange things in time.
And in a sense, improvisatory
and improvisatory movement
are both ways of--
different blocks, if you will,
in time as opposed to space.
PROFESSOR: Mm-hmm.
Yeah, my friends who seem
amusical, they probably--
maybe there's something
different about their cochlea.
Or maybe they have
absolute pitch
in some sense, which
is a bad thing to have.
Because if you're listening
to a piece composed
by a composer who doesn't
have absolute pitch,
then you're reading
all sorts of things
into the music that
shouldn't be there.
And if you're-- and the
opposite would be true.
I read music
criticism sometimes.
And maybe the reviewer says,
and after the second and third
movement, he finally
returns to the initial key
of E flat major--
what a relief.
Well, I once had absolute
pitch for a couple of weeks,
because I ran a tuning fork
in my room for a month.
And I didn't like it.
[LAUGHTER]
Because you can't
listen to Bach anymore.
Oh, well.
It's a good question,
why do people like music.
And I don't know any
other paper like mine.
If you ever find one,
I'd like to see it.
Because if you go
to a big library,
there are thousands
of books about music.
And if you open one, it's
mostly Berlioz complaining
that somebody wouldn't
give him enough money
to hire a big enough chorus.
But I've found very few
books about music itself.
Yes?
AUDIENCE: It's not
about music anymore.
Is that all right?
OK.
Do you think that having a
body is a necessary component
of having a mind?
Could you do just
as well just as a--
you know, sort of a
simulated creature?
PROFESSOR: Oh, sure, you could--
AUDIENCE: And have
all the things?
PROFESSOR: Simulation-- I think
a mind that's not in a world
wouldn't have much to do.
It would have to
invent the world.
And I don't see why it couldn't.
But you might have to give it
a start, like the idea of three
or four dimensions.
But can't you-- what happens
if you sit back and just
think for a while?
You wouldn't know if your
body had disappeared for--
would you?
There also is a strange
idea about existence and--
why do you think
there's a world?
One of the things
that bugs me is people
say, well, who created it?
And that can't make any sense.
Because this is just
a possible world.
Suppose there are a whole
lot of possible worlds,
and there's one real one.
How could you ever--
how could you possibly
know which one you're in?
And then you could say, well,
didn't someone have to make it?
And what's the next
thing you'd ask?
Well, who made the maker?
So the body-mind thing--
seems to me that once
you have a computer,
it can be its own world.
It just can sit.
The program can
spend half the time
simulating a world
and half the time
thinking about what
it's like to be in it.
Yeah?
AUDIENCE: Do you have a
current theory of existence?
PROFESSOR: Yes, it's
an empty concept.
It's all right to say
this bottle exists,
because that's saying this
bottle is in this universe.
But what would it mean to
say the universe exists?
The universe is in the universe?
So there's something wrong
with thinking about--
so there are only
possible worlds.
There's no-- it doesn't make
any sense to pick one of them
out and say that's the real one.
Yeah?
AUDIENCE: So it's
existence is relative?
PROFESSOR: Yeah, you don't
say, this is the world I'm in.
But you shouldn't say--
that doesn't mean it exists.
Like, 2 is in the
set of even numbers.
But what's the set
of even numbers in?
It doesn't stop anywhere.
Yeah?
Lots of worlds.
AUDIENCE: So is mathematics
[INAUDIBLE] worlds.
But physics, it only
explains the current world?
Or I don't know how you
compare these two subjects.
PROFESSOR: Well, you can't tell.
Because five minutes from
now, everything might change.
So nothing ever
explains anything.
You just have to
take what you've got
and make the best of it.
Yeah?
AUDIENCE: So this relationship
means systems knowledge
in artificial intelligence?
PROFESSOR: Which knowledge?
AUDIENCE: Systems,
basically, in general.
PROFESSOR: Well,
there are people
who talk about systems
theory, but I'm not sure
that it's well-defined.
AUDIENCE: Right, exactly.
PROFESSOR: Artificial
intelligence
means, to me, making a system
that is resourceful and doesn't
get stuck.
And so if you have a system--
and also, it's an--
how do you put it?
Some definitions
are not stationary,
like what's popular.
Popular is what's popular now.
There isn't any such
thing as popular music
in terms of the music.
So I know there were--
there was once a
little department
called systems analysis
at Tufts, which
had a couple of rather
good philosophers
try to make general
theory of everything.
And they were writing
nice little papers.
And it got a--
it moved along.
But then there was
this Senator McCarthy
you've probably heard of.
And he announced that he
had evidence that the--
one of the principal
investigators
had slept with his wife
before they were married.
Well, Tufts was very
frightened at this
and abolished that department.
And Bill Schutz went to
California, and started.
Esalen, and had a good time
for the next 50 years--
more stories.
Yeah?
AUDIENCE: Kind of as a extension
of the body-mind question,
it seems to me we--
as humans, we--
learn a lot from
just interacting
with the environment.
Like language, we
hear it being spoken.
We speak it.
We see things.
We touch things.
But as far as I know,
a lot of the efforts
in artificial
intelligence so far
have been confined to the
computer that does not go out
into the real world, interact.
Doesn't [INAUDIBLE] to
see, learn new think.
PROFESSOR: Well,
here's the problem--
I look over at Carnegie
Mellon, and there
are some nice projects.
And the most popular
one is robot soccer.
And here are these little
robots kicking a ball around.
They're Sony-- what
are they called?
AUDIENCE: AIBOs.
PROFESSOR: Yes, the Sony AIBOs.
Sony stopped making the AIBOs.
But it respected
Carnegie, and it
made a little stash,
secret stash, of AIBOs
to send to Carnegie when
the present ones break.
But my impression of AI projects
that have robots is that they
do less, less, less than
projects that don't.
The reason is, if you have a
robot like ASIMO, made by Sony?
No.
AUDIENCE: Honda.
PROFESSOR: Honda.
ASIMO can get in the back seat
of a car with some effort,
and usually falls over.
However, if you
simulate a stick figure
in a computer getting into
a stick figure of a car,
then you can make it learn to do
that and get better and better.
And so all AI projects
without robots
are way ahead of all AI
projects with robots.
And the profound reason is that
robots are usually expensive,
and they're always being fixed.
So if you have five students,
and the robot is being fixed--
I don't know what
they're doing--
but they have to wait.
Whereas if you have
a stick figure robot,
then you can just
run it on this,
although it might be a little
slower than your mainframe--
probably not.
Yeah?
AUDIENCE: So then back to the
idea of the mind and the body,
here's a theory that
I just thought of--
the idea of the body as a
seeming abstract, basically
a mechanism for input-output.
It's a set of sensors from which
our brains can get information
about the world, and
instead of actuators,
in which we can
display our state.
So in that light, it's almost
as if our brains are really
independent on our body itself.
It can adapt to any
sort of body if we
happen to hook it up that way.
And it just so
happens that we've
been hooked up to
this body since birth,
that we have such good mental
models of how to use this body.
And I guess an example of--
from experiments that support
this theory might be how,
when people have
limbs amputated,
it takes them a while to
forget that they have the limb.
Because their mental
model still exists.
The mental models don't
go away overnight.
And also, I guess, they train
monkeys to control robot arms
with their brains.
PROFESSOR: Sure.
Well, but it just seems
to me that a large amount
of our brain is involved with
highly evolved locomotion
mechanisms.
And as I said, when you're
sitting back with your eyes
closed in a chair
thinking about something,
then it's not clear how much
of that machinery is important.
But it might be that--
I have a strange paper on--
I don't know if it's--
trying to remember its name.
It's called-- do you think
I can actually get a--
I can't remember the
name of the title.
Oh, I give up.
The idea is that maybe--
in the older theories
of psychology,
everything is learned by
experience in the real world--
so conditioning, and
reinforcement, and so forth.
In this theory I call internal
grounding, I make a conjecture.
Suppose the brain has a little
piece of nerve tissue which
consists of a few neurons
arranged to make not
a flip-flop, but a--
what would you call a three-
or four-flop, a flip-flop
with three or four states?
Let's say three.
So when you put a certain
input, it goes from--
couldn't find the chalk.
So here are three states.
And here's a certain input that
means if you're in that state,
you go to this.
And if you pop that input
again, it does this.
And if you, say, go
counterclockwise, it goes.
So three of them get
you back where you were.
But if I go this,
this, and that,
that would mean to go
like this, this, and back.
So this would be--
that means that's equivalent
to just going one.
Get the idea?
In other words,
imagine that there's
a little world inside your
brain, which is very small
and only has three states.
And you have actions that
you can perform on it.
And you have an
inner eye which can
see which of the three points
of that triangle you're on.
Then you could
learn, by experience,
that if you go left, left, left,
you're back where you were.
But if you go left,
right, left, right,
you're back where you are.
And if you go left, left, right,
that's like going one left.
In other words,
you could imagine
a brain that starts out--
before it connects itself
to the real world,
it starts by having
the top level of
the brain connected
to a little internal world which
just has three or four states.
And you get very good
at manipulating that.
You add more sensory
systems to the outer world.
And you get to get--
learn ways to get around
in the real world.
So I call that the internal
grounding hypothesis.
And my suggestion is, maybe
somewhere in the human brain,
there's a little
structure that's
somewhat like
that, which is used
by the frontal
part of the cortex
to make very abstract ideas.
You understand?
The more abstract an idea is,
the simpler, and more stupid,
and elementary it is.
Abstract doesn't mean hard.
Abstract means stupid.
Real things like this are
infinitely complicated.
So we might have--
and I wouldn't dare suggest
this to a neuroscientist.
There might be some little
brain center somewhere
near the frontal cortex that
allows the frontal cortex
to do some predicting, and
planning, and induction
about a very simple--
a few simple,
finite-state arrangements.
Who knows?
Would you look for it?
Well, if you were
a neuroscientist,
you could say, oh,
that's completely
different from
anything I ever heard.
Let's look for it.
And if you're wrong,
you have wasted a year.
And if you're right, then you
become the new Ramon y Cajal
or someone.
Who's the best new--
who's the currently
best neuroscientist?
Maybe it's late.
One more question.
One last question.
This is Cynthia Solomon, who's
one of the great developers
of the Logo language.
Yay.
Yes?
AUDIENCE: So maybe it's a
bad question for the end.
I will ask anyway.
What do you think about theories
such as Rodney Brooks' theories
that can speak of no
central [INAUDIBLE]??
PROFESSOR: Completely weird.
Obviously, those
theories have nothing
to do with human thinking.
But they're very good
for making stupid robots.
And the vacuum cleaner is
one of the great achievements
of the century.
However, his projects--
what was it called?
Cog-- disappeared
without a trace.
That theory was so wrong
that it got a national award.
And it corrupted AI research
in Japan for several years.
I can't understand--
Brooks became popular
because he said,
maybe the important
things about thinking
is that there's no
internal representation.
You're just reacting
to situations.
And you have a big library of
how to react to each situation.
Well, David Hume had that idea.
And he was a popular philosopher
for hundreds of years.
But it went nowhere, and
it's gone, and so is Rod.
However, he is one of the
great robot designers.
And he may be
instrumental in fixing
the great Japanese
nuclear meltdown.
Because they're shipping
some of his robots out there.
The problem is, can
it open the door?
So far, no robot
can open the door
even though it's not locked.
[APPLAUSE]
Thank you.
