MICHALE FEE: So
today, we're going
to introduce a new
topic, which is related
to the idea of
fine-tuning curves,
and that is the notion
of receptive fields.
So most of you
have probably been,
at least those of you who've
taken 9.01 or 9.00 maybe,
have been exposed to the idea
of what a receptive field is.
The idea is basically
that in sensory systems
neurons receive input from
the sensory periphery,
and neurons generally have
some kind of sensory stimulus
that causes them to spike.
And so one of the
classic examples
of how to find receptive
fields comes from the work
of Huble and Wiesel.
So I'll show you
some movies made
from early experiments
of Huble-Wiesel
where they are recording in
the visual cortex of the cat.
So they place a
fine metal electrode
into a primary visual
cortex, and they present.
So then they anesthetize the
cat so the cat can't move.
They open the eye,
and the cat's now
looking at a screen that
looks like this, where
they play a visual stimulus.
And they actually did this with
essentially a slide projector
that they could
put a card in front
of that had a little
hole in it, for example,
that allowed a spot of light
to project onto the screen.
And then they can move
that spot of light
around while they record
from neurons in visual cortex
and present different visual
stimuli to the retina.
So here's what one of
those movies looks like.
So you're hearing
the actual potential
of a neuron visual cortex.
So you can see the
neuron generates
lots of spikes when you
turn a spot of light
on a particular part
of the visual field.
So they will basically play
around with spots of light
or bars of light and see where
the neuron spikes a light,
and then they would
draw on the screen--
I think they're going to
draw in a moment here--
what they would call the
receptive field of the neuron.
So you can see that this neuron
responds with high firing rate
when you turn on a stimulus
in that small region there.
Notice that the cell also
responds when you turn off
a stimulus that is
right in the area
surrounding that
small receptor field.
So the neuron has parts of
its receptive field that
respond with increased
firing when you apply light,
and they also have parts
of the receptive field that
respond with higher firing
rate when you remove light.
That was actually a
cell, I should have said,
that's in the thalamus that
projects to visual cortex.
So that was a thalamic neuron.
Here's what a neuron in
cortex might look like.
So they started recording
in the thalamus.
They saw that those neurons
responded to spots of light
in small parts of
the visual field.
They were actually
recording from neurons
in the visual cortex.
They got kind of--
they couldn't really
figure out what the
neurons were doing,
and they pulled the slide
out of the projector, which
made an edge of light moving
across the visual field.
And the neuron they were
recording from at that moment
responded robustly when
they pulled the slide out.
And they realized,
oh, maybe it's
an edge that the neuron
is responding to.
And so then they started doing
experiments with bars of light.
Here's an example.
So you can see the neuron
responds when you turn
a light on in this area here.
But is responds when you turn
light off in this area here.
And so you can see
they're marking
a symbol for positive responses,
positive responses to light
on here, and negative
responses or increased firing
when you turn the light off.
So there's different
parts of the receptive
field that have positive
and negative components.
But you can see that
the general picture here
is that the process of
finding receptive fields
at this early stage
was kind of random.
You just tried different
things and hoped
to make the neurons spike.
And we're going to come
back to this idea of finding
receptive fields by
trying random things,
but in a more systematic way,
at the end of the lecture today.
So here's what we're
going to be talking about.
So you can see that
Hubel and Wiesel were
able to describe that
receptive field by finding
positive and negative
parts and writing symbols
down on a screen.
We're going to take a
more mathematical approach
and think about what that means
in a quantitative model of how
neurons respond to stimuli.
And the basic model that
we'll be talking about
is called an LN model,
linear/nonlinear model.
And we're going to
describe neural responses
as a linear filter that acts on
the sensory stimulus followed
by a nonlinear function that
just says neurons can only
fire at positive rates.
So we're going to have our
neurons spike when that filter
output is positive, but not when
the filter output is negative.
And we're going to describe
spatial receptive fields
as a correlation
of the receptive
field with the stimulus.
And we're also going
to talk about the idea
of temporal receptive
fields, which
will be a convolution
of a temporal receptive
field with the stimulus.
So the firing rate
will be a convolution
of a receptive field with
the temporal structure
of the stimulus.
We're going to
then turn to the--
combine these things
into the concept
of a spatial temporal receptive
field that simultaneously
describes the
spatial sensitivity
and the temporal sensitivity
of a neuron, as an STRF,
as it's called.
And we'll talk about the
concept of separability.
And finally, we're going
to talk about the idea
of using random noise
to try to drive neurons,
to drive activity in
neurons, and using what's
called a spike-triggered average
to extract the stimulus that
makes a neuron spike.
And we're going to
use that to compute--
we're going to see how
to use that to compute
a spatial temporal receptive
field in the visual system
or a spectral temporal receptive
field in the auditory system.
So let's start with this.
What are spatial and
temporal receptive fields?
So we just saw how you can think
of a region of the visual space
that makes a neuron spike when
you turn light on or makes
a neuron spike when
you turn light off.
And at the simplest
level, you can
think of that in
the visual system
as just a part of
the visual field
that a neuron will respond to.
So if you flash of light over
here, the neuron might respond.
If you flash of light over
here, it won't respond.
So there's this region
of the visual field where
neurons respond, but it's
more than just a region.
There's actually a pattern of
features within that area that
will make a neuron
spike, and other patterns
will keep the
neuron from spiking.
And so we can think of a neuron
as having some spatial filter
that has positive
parts, and I'll
use green throughout
my lecture today
for positive parts
of a receptive
field, and negative parts.
And this is a
classic organization
of receptive fields,
let's say, in the retina
or in the thalamus, where you
have an excitatory central part
of a receptive field and
an inhibitory surround
of the receptive field.
So we can think of
this as a filter that
acts on the sensory input.
And the better the stimulus
overlaps with that filter,
the more the neuron will spike.
So let's formalize this a
little bit into a model.
So let's say we have
some visual stimulus that
is an intensity as a
function of position x and y.
We have sum filter, G,
that filters that stimulus.
So we put the stimulus
into this filter.
This filter, in this case,
just looks like, in this case,
an excitatory surround
in an inhibitory center.
That filter has an
output, L, which
is the response of the filter.
Then we have some nonlinearity.
So we take the response
of the filter, L,
we add it to some
spontaneous firing rate,
and we take the positive
part of that sum
and call that our firing rate.
So that would be a typical
output nonlinearity.
It's called "a
threshold nonlinearity,"
where if the sum of
the filter output
and the spontaneous firing
rate is greater than 0,
then that corresponds to the
firing rate of the neuron.
So in this case, you can see
that as a function of L--
I should have
labeled that axis L--
as a function of
L, when L is 0, you
can see the neuron has
some spontaneous firing
rate, R naught.
And if L is positive,
the rate goes up.
If L is negative, the rate goes
down until the rate hits 0.
And then once the neuron stops
firing, it can't go negative.
So the neuron firing
rate stays at 0.
And then once you have this
firing rate of the neuron, what
is it that actually determines
whether the neuron will spike?
So in most models
like this, there
is a probabilistic
spike generator
that is a function of the
rate output of this nonlinear
output.
It's basically a random process
that generates spikes at a rate
corresponding to this R.
And in the next
lecture, we're going
to come back and talk a lot more
about what spike trains look
like, how to characterize
their randomness,
and what different
kinds of random
processes you actually
see in neurons.
A very common one is
the Poisson process,
where there's an equal
probability per unit
time of a neuron
generating a spike,
and that probability is
controlled by the firing rate.
We'll come back to that
and discuss it more.
Any questions?
Yes, [INAUDIBLE]
AUDIENCE: Is something
biologically [INAUDIBLE]
something like if the overlap
[INAUDIBLE] it's just it's
more excitatory?
MICHALE FEE: Yeah.
So we're going to
come back in, I think,
a couple lectures
where we're going
to talk about exactly how you
would build a filter like this
in a simple feed
forward network.
So at the simplest
level, you can just
imagine you have a
sensory periphery that
has neurons in it that
detect, let's say,
light at different positions.
Those neurons send axons
that then impinge on,
let's say, the neuron that
we're modeling right here.
And the pattern of
those projections,
both excitatory and inhibitory
projections from the periphery,
would give you
this linear filter.
And then this nonlinearity would
be a property of this neuron
that we're modeling.
Yes, Jasmine?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Great, exactly.
So it's going to
turn out that we're
going to treat this
as a linear filter.
The output of this
filter will be calculated
for a spatial receptive field as
the correlation of this filter
with the stimulus.
But in the time domain, when we
calculate a temporal receptive
field, we're going
to use a convolution.
And we'll get to
that in a minute.
That's the very next thing.
Great question.
Anything else?
So that's called a LN model.
I should have put
that on the slide--
linear/nonlinear model.
So let's describe
that mathematically.
So let's say we have a
two-dimensional receptive
field.
We're going to call
that G of x and y.
So remember, we had intensity
as a function of x and y.
There's our stimulus input.
And we're going to ask, how
well does that stimulus overlap
with this receptive field?
And we're going to describe the
receptive field as a function
on this space, x and y.
And our linear
model is going to be
how well the stimulus
matches or overlaps
with the receptive field.
And we do that just by
multiplying the receptive
field times the stimulus and
integrating over x and y.
x and y, just think of it
as a position on the retina.
So let's look at this
in one dimension.
So remember, this
was a receptive
field that has a
positive central region
and an inhibitory surround.
So if we just take
a slice through that
and plot G as a
function of x, you
can see that there is a positive
central lobe and inhibitory
surround, inhibitory side load.
That's a very, very common
receptive field early
in the visual
system, in the retina
and in the lateral
geniculate nucleus.
So in one dimension, we just
take this receptive field, G,
multiply it by the
stimulus pattern,
and integrate over position.
That is L. That's the
output of the linear filter.
We're going to add that to
a spontaneous firing rate,
and that gives us the
firing rate of our neuron.
And you can see
that that's like--
that this product,
an integral over x,
is just like a correlation--
G of I times intensity
of I summed over I.
So let's walk through
what that looks like.
So here's G of x,
the receptive field.
Let's say that's our
intensity profile.
So we're going to have a
bright spot of light surrounded
by a darker side lobe.
So the way to think
about this is,
in visual neuroscience
experiments, usually
the background is kind of gray.
And you'll have bright spots,
like here, and dark spots,
like there.
And the rest will just be gray.
So that's how you get positive
and negative intensities here,
because they're relative to
some kind of gray background.
And so now we can just
multiply those two together.
And you can see that when
you multiply positive times
positive, you get positive.
And when you multiply
negative times negative,
you get positive.
And when you integrate
over position x,
you get a big number.
You get some positive number.
So that neuron,
that stimulus, would
make this a neuron
with this receptive
field likely to spike.
Let's consider this case.
Now, instead of a small
spot of light centered
over the excitatory lobe
of the receptive field,
you have a broad
spot of light that
covers both the excitatory
and inhibitory lobes
of the receptive field.
What's going to happen?
Yeah?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
You're going to get a positive
times of positive here,
and a negative times a positive.
And so you're going to
get negative contribution
from the side lobes,
and those things
can exactly cancel out when
you integrate over position.
And so you can get a
very small response.
If you have--
I'm not going to go
through this example,
but it's pretty obvious.
If you were to have
light here, and then dark
here, and then light
there, you would
have only these negative
side lobes activated.
You would have no contribution
from this excitatory lobe,
and the integral would
actually be negative.
And so the firing rate of
the neuron would go down.
Do you remember seeing
those different points
in that first
movie where you saw
the donut of light turning on?
The neuron kind of shuts off
when you turn that light on.
Yes?
AUDIENCE: So I'm
just curious what
we're signing up 0 [INAUDIBLE].
MICHALE FEE: Yeah.
So 0 is just this
gray background.
It's some intermediate
level of light intensity--
pretty straightforward.
So that's a spatial
receptive field right there.
We refer to this correlation
process, this linear filter
as linear, because you
can see that if you
put in half the light
intensity, let's say,
you get half the product.
And when you integrate, you
get half the neural response.
If you take the stimulus and you
cut it in half so that you only
apply light and dark to
half the receptive field,
then you'll also get half
the response of the neuron.
Because this will contribute
to the integral and this
won't, and so you'll
get a neural response
that's half as big.
So in this model, the
response varies linearly
with this overlap of
the receptive field
and the intensity.
So that's where the
term linear comes from.
Any question about that?
Correlation is a
linear operation.
So the next thing we're
going to talk about
is temporal receptive field.
So we just talked about
spatial receptive fields.
Neurons are also very sensitive
to how things vary in time.
So we're going to
take the same concept.
Instead of a stimulus that's
a function of position
on the retina, let's
say, we're going
to take a stimulus that's
a function of time.
And we're going to operate
on that temporal stimulus
with a filter that [INAUDIBLE]
a temporal sensitivity.
We're going to get the
output of that filter,
add it to a spontaneous
firing rate,
and we're going to get a
time-dependent firing rate.
So let me just show you
what this looks like.
So let's say that you
have a stimulus that's
fluctuating in time.
So imagine that you
have a neuron that
has a spatial receptive
field that's just a blob,
and you shine just
a positive bump.
And you shine light on it, and
the intensity of that light
varies.
And this is the
intensity that you
apply to that spatial receptive
field as a function of time.
And again, 0 is some kind
of average gray level.
And so you can go dark,
dark, or bright around that.
So now, neurons generally
have receptive fields in time,
and this is what a typical
receptive field might
look like, a temporal receptive
field might look like,
for a neuron.
Neurons are often
particularly driven
by stimuli that go dark
briefly and then go
bright very suddenly, and
that causes a neuron to spike.
So we can imagine that
temporal receptive field,
and sliding it
across the stimulus,
and measuring the overlap of
that temporal receptive field
with the stimulus at each time.
Does that makes sense?
And so you can see
that most of the time
that negative bump
and positive bump
are just going to be
overlapping with just
lots of random wiggles.
But let's say that
the stimulus has
a negative part and
then a positive part,
dark, then bright.
You can see that at
this point that filter
will have a strong
overlap with the stimulus.
Why?
Because the negative overlaps
with the negative, and that
product is positive.
Positive overlaps with positive,
that product is positive.
And so when you integrate
over time you get peak.
Does that make sense?
This is, what I'm plotting here,
is the product of these two
functions at each
different time step
as I slide this
temporal receptive
field over the stimulus.
Does that make sense?
Any questions about that?
I'm going to go through that.
We're going to work on
that idea little bit more,
but if you have any
questions, now's a good time.
AUDIENCE: [INAUDIBLE]
MICHALE FEE: So
you're asking, why
is it correlation in
the spatial domain?
Because-- well, let me
answer that question
after we define what
this is mathematically.
So what is this mathematically?
AUDIENCE: A convolution.
MICHALE FEE: It's a convolution.
It's exactly what we talked--
it's a lot like what we talked
about when we talked synapses.
In that case, we had
some delta functions here
corresponding to
spikes coming in,
and the synaptic response was
like some decaying exponential.
And we slid that
over the stimulus.
In this case, we have this
fluctuating sensory input,
this light intensity, and we're
sliding that linear response
of the neuron over and
measuring the overlap
as a function of position,
and that's a convolution.
Mathematically,
what we're doing is
we're taking this linear
kernel, this linear filter,
sliding it over the stimulus,
using this variable t,
and we're integrating
over this variable tau.
So we have a kernel, D,
multiplied by the stimulus
at different times shifts.
We integrate over
tau, and that's
the output of our
temporal receptive field.
That's the linear output
of our receptive field.
And we add that to
spontaneous firing right,
and that gives us a
time-dependent firing
rate of the neuron.
Yes?
AUDIENCE: So is tau how
much we [INAUDIBLE]??
MICHALE FEE: Great question.
t is the location of this kernel
as we're sliding it along.
Tau is the variable that
we're integrating over
after we multiply them.
Does that make sense?
So we're going to pick a
t, place the kernel down
at that time, multiply this.
And remember, this is 0
everywhere outside of here.
And so we're going to multiply
the stimulus by this kernel.
It's going to be-- that product
is going to be 0 everywhere
except right in here.
You're going to get
a positive bump when
you multiply these two,
a positive bump when
you multiply those two.
And the integral over tau gives
us this positive peak here.
If we picked a
slightly different t
so that this thing was lined up
with this positive peak here,
then you'd see that you'd
have positive times negative.
That gives you a negative.
When you integrate over
tau, that gives you
this negative peak here.
Does that makes sense?
So let's just go
back to the math.
So you can see that we're
integrating over tau,
but we're sliding the
relative position of D and S
with this variable t.
Yes?
AUDIENCE: Is that
the [INAUDIBLE]??
MICHALE FEE: Yes.
S is the stimulus.
AUDIENCE: Oh.
And D is the kernel?
MICHALE FEE: D is
the linear kernel.
Yes?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: So nature chooses
the shape of the kernel for us.
So that is the receptive
field of neurons.
Now, I just made this
up to demonstrate
what this process looks like.
But in real life, this is
the property of a neuron,
and we're going
to figure out how
to extract this property
from neurons using
a technique called
spike-triggered average, which
we'll get to later.
But for now, what
I'm trying to convey
is, if we knew this temporal
receptive field of a neuron,
then we could predict the
firing rate of the neuron
to a time varying stimulus.
That was a very
important question.
Does everyone understand that?
Because it's one of those
cases where once you see it
it's pretty obvious,
but sometimes I
don't explain it well enough.
Yes?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yes.
So I've already flipped it,
and sometimes you'll see.
So this is all going
this way-- positive tau.
I've flipped it for you already.
Sometimes you'll see it
plotted the other way with tau
going positive to
the right, but I've
plotted it this way already.
Any questions?
Oh, and so that was actually
the very next question.
You might normally--
you might sometimes
see temporal receptive
fields plotted
this way with positive
tau going to the right.
And kind of meant--
I always just flip it back over.
Because in this view, you see
that what the neuron responds
to is dark followed by
light, and then right there
is when you have a peak
spiking probability.
Peak firing rate
happens right here.
Yes?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: So we're
going to get to that.
But typically, neurons
in the retina--
I'll show you an
example in the retina.
A typical time scale here might
be tens to 100 milliseconds,
so pretty fast.
So that's called
the temporal kernel
or the temporal receptive field.
And again, it's
linear in the sense
that if you, for example, had
a stimulus intensity that just
had this positive bump without
the negative bump, then
the response would be lower
just by the ratio of areas.
So if you got rid of this
big negative bump here,
then the response would be,
I don't know, a third as big.
It would be linear in the area.
Let's push on.
So now, let's extend this.
So we've been talking about
spatial receptive fields
and temporal receptive fields.
But in reality, you can
combine those things together
into a single concept, called
a "spatial temporal receptive
field," and that's usually
referred to as an STRF.
If you're working in the
auditory system, STRF,
it's the same
acronym, but it just
means spectral temporal
receptive field,
because it's sensitive
to the spectral content
of the sounds, not
the spatial structure
of the visual stimulus.
So in general, when you
have a visual stimulus,
it actually depends on
x- and y-coordinates
in the retina and time.
So just I of x and y, which
would be like a still image
presented to you.
I of x, y, and t is--
any movie can be
written like that.
Your favorite movie is just some
function of I of x, y, and t.
And so we're going to now
present to our retina,
and we're going to simplify
this by considering
just one spatial dimension.
So we're going to take
your favorite movie
and just collapse
it into intensity
as a function of position.
It's probably not
nearly as interesting,
but it's much easier to analyze.
So we're going to
write the firing rate
as a function of time
as a spontaneous firing
rate plus a filter, D, which
is a spatial temporal receptive
field acting on that intensity.
And you can see that we're
doing stuff in here that
looks like a convolution
integrating over tau,
and we're also doing stuff that
looks like a correlation when
we integrate over x.
So there's the convolution
integrating over tau.
What I've done is I've
pulled out the D tau,
because we can consider--
I've just written this as
two separate integrals.
So we have an integral over tau
that looks like a convolution.
And we have an integral over x
that looks like a correlation.
So what is separability mean?
So separability is
just a particularly--
if a receptive
field is separable,
it means that you can write
down a spatial receptive field
and a temporal receptive
field separately.
And that looks like this.
So I imagine that if you have
a spatial temporal receptive
field, D, that's a function
of position and time.
But you can see
that you can just
write it as a product
of the spatial part
and the temporal part.
So here, you have a
temporal receptive field
that looks like this, a positive
lobe here and a negative lobe
there, a spatial receptive
field that looks like this, just
a positive lobe.
And if you multiply
this function
of x by this function
of t, you can
see that you get a
function of x and t
that looks like this,
where at any position
the function of time just
looks like this-- scaled.
And at any time, the
spatial receptive field
just looks like this.
Does that make sense?
Other receptive fields
are not separable.
You can see that you
can't write this receptive
field as a product of a
temporal receptive field
and a spatial receptive field.
Does that make sense?
Is that clear why that is?
So basically, you can see
that if you take a slice here
at a particular
position, you can
see that the
temporal pattern here
looks very different than
the temporal pattern here.
And so you can't
write this simply
as a product of a spatial and
a temporal receptive field--
separable, inseparable.
So let's take a look at
what happens when you have
a separable receptive field.
Things kind of
become very simple.
We can now write our spatial
temporal receptive field
as a spatial
receptive field, which
is a function of position,
times a temporal receptive field
that's a function of time.
And when you put that into
this integral, what you find
is that you can pull that
spatial part of the receptive
field out of the
temporal integral.
So basically, the way
you think about this
is that you find the correlation
of the spatial receptive field
with the stimulus,
and that gives you
a time-dependent stimulus,
a stimulus that's
just a function of time.
Then you can convolve the
temporal receptive field
with that
time-dependent stimulus.
So you can really just treat
it as two separate processes,
which can be kind
of convenient just
for thinking about
how a neuron will
respond to different stimuli.
So let's just think about,
develop some intuition about,
how neurons with a
particular receptive field
will respond to a
particular stimulus.
So here's what I've done.
I've taken a spatial temporal
receptive field here.
This is a function
of position and time,
and we're going to figure
out how that neuron responds
to this stimulus.
So this stimulus is also a
function of space and time.
It's one-dimensional in space.
So what does this look like?
This looks like a bar of light
that extends from position 2,
let's say, 2 millimeters to
4 millimeters on our screen.
And it turns on at
time point 1, stays on,
and turns off at time point 6.
Let's say 1 second to 6 seconds.
Does that make sense?
So just imagine we have
a 1D screen, just a bar,
and we turn on light that's
a bar between 2 and 4.
So we turn on a bar of light.
We turn it on at time 1, and
we turn it off at time 6.
It's just a very simple case.
We flash of light on at
a particular position,
and then we turn it off.
So let's see how
this neuron responds.
So what we're going to do
is we're going to slide--
remember, in the 1D case where
we had the temporal receptive
field, we just slid it
across the stimulus.
So we're going to do
the same thing here.
We're going to take that spatial
temporal receptive field,
and we're going to slide
it across the stimulus.
And we're going to
integrate, we're
going to take the product,
and we're going to integrate.
And the integral plus
the spontaneous rate
is going to be the firing
rate of our neuron.
So what is the
integral right there?
The product is--
AUDIENCE: 0.
MICHALE FEE: --0.
where? integrate, it's 0.
So we're going to add a
spontaneous firing rate, which
will be right there.
So that will be our firing rate.
Now, let's slide the
stimulus a little further.
That means that this
time we're asking,
what is the firing
rate of that neuron?
So what is it
going to look like?
AUDIENCE: Go up a bit.
MICHALE FEE: It's going
to go up a little bit,
because we have a positive
part of the receptive field.
Green is positive in
our pictures here.
It's going to overlap
with this bar of light,
because that neuron is sensitive
to light between, let's say,
1 and 4, positions 1
and 4 on the screen.
And so the light is falling
within the positive part
of that receptive field,
and so the neuron's
going to increase
its firing rate.
So now what's going to happen?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: It's
going to cancel.
You're going to get a positive
contribution to the firing rate
here--
whoops-- and a negative
contribution here.
And those two are
going to add up.
You're going to multiply
that times that.
That gives you a plus.
That times that
gives you-- sorry.
That times the light that's
shining on it is negative.
Add those up, and
it's going to cancel.
And the firing rate's going
to go back to baseline.
Now, the light in
this receptive field,
we're continuing to slide it
in time over our stimulus.
What happens here?
AUDIENCE: Same thing.
MICHALE FEE: Same, good.
How about here?
It's going to go--
AUDIENCE: Down.
MICHALE FEE: --down.
It's going to dip
down, that's right.
And then?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
By 0, you mean the
spontaneous firing.
Yeah, exactly.
Cool.
Yes?
AUDIENCE: [INAUDIBLE] the rate
of response because the slope
of the line [INAUDIBLE]?
MICHALE FEE: So you should
think about this thing sliding
over the stimulus in real time.
So if these are
units of seconds,
then this thing is sliding
across the stimulus
at 1 second per
second sliding across.
And so that is firing
rate as a function
of time in those units.
Does that make sense?
AUDIENCE: Yeah.
But why [INAUDIBLE]?
MICHALE FEE: Oh, OK.
Like why doesn't
this go up to here?
So what's the answer to that?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
So how would I
make that steeper?
How would I make
that go up to here?
AUDIENCE: [INAUDIBLE] the light.
MICHALE FEE: What's that?
AUDIENCE: You'd
turn the light up.
MICHALE FEE: Yeah, you'd turn
the light up, that's right.
This is the receptive
field of a neuron,
so we generally
can't control that.
So if we wanted to make
this neuron respond more,
we'd turn the light up
to a higher intensity.
Any other questions?
So neurons often have more
complex receptive fields.
So here's an example.
What is this going to do as we
slide this across the stimulus?
What's that?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
It's not going to--
the neuron isn't going
to respond at all.
Because as soon as it overlaps,
it has a positive contribution.
The light activates these
lobes of the receptive field,
but inhibits these lobes
of the receptive field.
And the net result is that when
you integrate over the product,
you're going to get 0.
Does anyone have any idea
what kind of stimulus
might make this neuron respond?
This is a very special
kind of receptive field.
Yes, [INAUDIBLE]
AUDIENCE: The light
goes from [INAUDIBLE]
MICHALE FEE: Yeah.
What is that called?
AUDIENCE: I'm not sure.
MICHALE FEE: It's called
a stimulus that moves.
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Moves--
a moving stimulus.
Good.
So that's a receptive field that
response to a moving stimulus.
So let's take a look at that.
So here we go.
Anybody want to take a guess
at what this stimulus will
do to this neuron?
Can you visualize
sliding it across?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: And then what?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
You can see that this--
so let's describe
what this [AUDIO OUT]..
So we've turned a bar of
light on here between 0 and 2,
and then we slide it up over
the course of a few seconds.
So we've turned a spot of light
on, and then we move it up--
off.
So it's a spot of light that
turns on, moves, and then
disappears.
So let's walk through it.
So there's a little
bit of overlap
there, so the
neuron's firing rate
is going to start going up.
But then as it goes
further, this light
is now activating
those inhibitory lobes,
which is going to have
a negative contribution.
So when you take
the product, you're
going to get lots of negatives
there, very little contribution
from the positive lobes,
and so the firing rate's
going to go down.
And what happens
is it goes down,
and once the firing rate hits 0,
it can't go any more negative.
So the firing rate is
just going to sit at 0
until this stimulus moves
out of the temporal receptor
of this neuron.
And then what?
AUDIENCE: Back up.
MICHALE FEE: It's
going to go back up
to the spontaneous rate.
So what kind of stimulus
will activate this neuron?
A stimulus that
moves from top down.
So let's take a look at that.
So here's our stimulus.
You see that it's going to
just hit that inhibitory lobe,
go down a little bit.
And then the excitatory
lobes of the receptive field
are going to overlap
with the stimulus.
You're going to get
a big positive peak,
and then the stimulus will move
out of the receptive field,
and the firing rate will
go back down to baseline.
Any questions about that?
So that's very common,
in both the visual system
and in the auditory
system, to have
neurons that are responsive
to moving stimuli.
What does moving stimulus
mean in the auditory system?
AUDIENCE: Changing pitch.
MICHALE FEE: Right,
changing pitch.
So [WHISTLE], like that.
That activated a
gazillion neurons
in your brain that are sensitive
to upward-going pitches
that you have
structure like this.
Isn't that crazy?
[WHISTLE]
I can control all the
neurons in your brain--
[WHISTLE]
[LAUGHS]
--at least the ones that
respond to whistles.
So now that we've
seen mathematically
how to think about what
a receptive field is
and how it interacts
with a sensory stimulus,
how do you actually
discover what the receptive
field of a neuron is?
That turns out to actually be
a very challenging problem.
So in very early
parts of the visual
and the auditory system, like
in the retina and the LGN,
and as far as, let's
say, V1 in visual cortex,
it's been possible to find
receptive fields of neurons
by basically just randomly
flashing bars and dots of light
and just hoping to get lucky
and find what the response is.
It turns out that that's
generally a very--
it can be a very
time-consuming process.
And so people have
worked out ways
of discovering the
receptive fields of neurons
in a much more systematic way.
And that's what we're going
to talk about next-- the idea
of a spike-triggered average.
So here's the idea.
We're going to take a stimulus,
and we're going to basically--
we're basically just
going to make noise,
just a very noisy stimulus.
So we're going to take, let's
say, an intensity, a light,
a spot of light, and we're
going to fluctuate the intensity
of that light very rapidly.
And we're going to do that
basically with a computer.
We just take a computer, make
a random number generator,
hook that up to, let's
say, a light source
that we can control the
brightness of with a voltage.
And then have the
computer generate--
put out that random
number sequence,
control the light level,
and then play that to our,
let's say, our visual neuron.
And that neuron
is going to spike.
And now, what we can do is
take the times of those spikes
and go back figure
out basically what
made the neuron fire post-hoc.
So if we do that here, you can
basically take the spike times.
Now, you know that whatever
made the neuron spike happened
before the spike.
It didn't happen
after the spike.
So you can basically
ignore whatever
happened after the spike and
just consider the stimulus that
came in prior to the spike.
So we're just going to take a
little block of the stimulus
prior to the spike,
and we're going
to do that for every spike
that the neuron generates,
and we're going to pile those
up and take an average--
spike-triggered.
That's it.
And that is going to be--
what's really cool is
that you can show that
under some conditions that
spike-triggered average is
actually just the receptive
field of the neuron.
It's the linear receptive
field of the neuron.
So let's think-- and you can
write that down as follows.
We're going to add a stimulus.
We're going to write down the
times at which all these spikes
occur, t sub i, or the
times in the stimulus
at which the spikes occur.
We're going to take the stimulus
at those times minus some tau,
and we're going to average them
over all the spikes, all the n
spikes, that we've measured.
And that K of tau is going to
be the spike-triggered average,
and in many cases, it's
actually the linear kernel.
Now, let's think for a moment
about what the conditions are.
What kind of
stimulus do you have
to use in order to get the
spike-triggered average
to actually be the linear kernel
of the neuron, that receptive
field of the neuron?
Any guesses?
Let me give you a hint.
What happens if I
take a stimulus that
varies very slowly?
So instead of having
these wiggles,
it just goes like this.
It has very slow,
random wiggles.
Will that be a good stimulus for
extracting the receptive field?
Why is that?
Yes?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah.
So I think what you're saying
is that that stimulus is very
slow, and it doesn't
actually have
the fast fluctuations in it
that makes the neuron spike.
If the stimulus varies
very slowly, then it--
see, this neuron likes to
have this very fast wiggle,
this negative followed
by a positive.
But if the stimulus you
put in just varies slowly,
then that stimulus
doesn't actually
have the kind of signal that's
needed to activate this neuron.
Yes?
AUDIENCE: [INAUDIBLE]
the stimulus [INAUDIBLE]
smaller than tau?
MICHALE FEE: Well, tau is just
the variable that describes
the temporal receptive field.
But I think what you're saying
is that the stimulus varies
more slowly than the
temporal structure
in the receptive field, in
the temporal receptive field.
That's right.
Tau is just this variable that
we define the receptive field
on.
Yes, [INAUDIBLE]
AUDIENCE: So when we
add up an average,
are we actually adding
up from everything
before the spike,
everything before the spike?
MICHALE FEE: So great question.
So how far back do you think
you would need to average?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Maybe.
I mean, in principle, you
could have spikes happening
very fast, and you
could have signal
that affects the response
of a neuron from even
before the last spike.
But in general, what
do you think the answer
to that question is?
Brainstorm some more ideas.
So let's say that you were
recording in the retina,
and you knew that neurons tend
to respond to visual stimuli
only for--
that temporal receptive
fields in the retina
never extend back more
than 100 milliseconds.
Then how would you
choose that window?
You would just choose this
window to be 100 milliseconds,
and that would be it.
If you're recording
in a brain area
that you really have
no idea, then you
have to actually try
different things.
So you can try a window that
goes back 100 milliseconds.
And if when you do the
spike-triggered average,
it hasn't gone to 0 yet, then
you need to take more window.
So you can figure it out.
You can create a short
window, and it only
takes-- like you change one
number in your Matlab code,
and hit Run again,
and do it again.
It's pretty simple.
Yes?
AUDIENCE: So when you've
got like [INAUDIBLE]..
MICHALE FEE: Yes.
AUDIENCE: Wouldn't
that depend [INAUDIBLE]
what kind of filter [INAUDIBLE]
MICHALE FEE: Yeah.
So you're saying that the
stimulus that you choose
actually depends on the kinds
of filters that the neurons are,
right?
Actually, the right
answer, it depends.
AUDIENCE: And so we have--
[INAUDIBLE]
MICHALE FEE: Yeah.
So generally, the statement
is that the stimulus you use
has to have fluctuations
in it that are faster
than the fluctuations
in the kernel
that you're trying to measure.
And so most people choose
what's called a "white noise
stimulus."
And white noise stimulus
comes from the idea
that when you take
the spectrum--
and we're going to get
into spectra next week.
But when you look at the
spectrum of the stimulus,
and you take the
Fourier transform of it
and look at how much power there
is as a function of frequency,
the spectrum is flat.
And just like in colors, white
light has a flat spectrum.
And so the term evolved to a
noise that has a flat spectrum
white noise.
And that's what people
generally refer to when they
do spike-triggered averages.
They use noise that
has a flat spectrum.
And so you'll often
refer to people
saying that they've used a
white noise stimulus to extract
a spike-triggered average.
Now, of course, you
can't ever make a noise
that truly has a flat spectrum.
You have to-- you
can only make things
fluctuate as fast as your
experimental setup can
make them fluctuate.
So things eventually fall off.
Fortunately, neurons tend to
have receptive fields that
only have fluctuations
that are on the scale of 10
milliseconds, or maybe a
millisecond in extreme cases.
Maybe in the auditory system you
might get a little fluctuation
for a millisecond, and
that's generally pretty
slow for an experimental setup.
So you choose a
white noise stimulus,
where white noise means
it's got fluctuations
faster than the
fastest fluctuations
in the temporal receptive field,
and that for real neurons tends
to be, even in
early sensory areas,
tends to be on the scale of
millisecond fluctuations.
And in higher brain
areas, they would
have even slower fluctuations.
Now, let me just say one word
about spike-triggered average.
It works really well
in lower sensor areas.
But once you get up out
of primary sensory areas,
this method doesn't
work any more.
Neurons don't actually have
simple receptive fields.
And this method starts not
working so well outside
of primary sensory areas.
But before I get too much
into the limitations of this,
let me just show you
some examples where it
works really beautifully well.
So this is--
I'll show you some slides that
I got from Marcus Meister, who
studies--
he's at Caltech.
He used to be at Harvard,
and he studies the retina.
And so he developed this setup
for extracting receptive fields
of retinal neurons.
And here's the idea.
So here's a piece of retina,
thats a representation
of the circuitry in the cells
within a piece of retina.
You extract the retina,
and you place it
on a dish, a special dish,
that has electrodes embedded,
metal electrodes
embedded, in the glass,
sort of on the
surface of the glass.
You take the retina out.
You press it down
onto the glass.
So now the electrodes are
sensing the spiking activity
of these neurons down here
in the retinal ganglion cell
layer.
These are the
photoreceptors up here.
And then what he does is he
has a computer monitor that's
generating random patterns
of visual stimuli,
and you project that
using a lens down
onto the photoreceptors
of the retina.
And those neurons now
make lots of spikes,
and you can extract those spikes
using the methods that you
saw in the video from Tuesday.
So here's what those
signals look like.
This just shows the signal
on four different electrodes
that happen to be
right near each other,
like four adjacent electrodes
on this electrode array.
And you can see that you
get spike wave forms on all
these different electrodes.
You can see that you get--
that you see what looks like
lots of cells on these four
electrodes.
One really interesting
thing to note
is that these electrodes are
actually placed close enough
together that multiple
electrodes detect
the spike signal
from a single cell.
So you can see right
here, here's a spike.
It exactly lines up with the
spike on this other electrode.
So here is a spike on one
electrode that lines up
with a spike on
another electrode.
You can see there's
a little blip there
and a little blip there.
All of those spikes are
actually from a single cell
whose electrical
activity is picked up
on four adjacent electrodes.
David?
AUDIENCE: Is this the raw data?
MICHALE FEE: Yeah.
This is the raw voltage data
coming out of those electrodes.
And you can see here's at a
different cell right here.
You can see that this cell has
a peak of voltage fluctuation
on electrode two.
You see a little blip there,
and a little blip there,
and nothing there.
And here is yet
another cell that
has a big peak on electrode
three, essentially nothing,
maybe a small bump there.
So you can actually extract
many different cells
looking at the patterns
of activity that
appear on nearby electrodes.
And it turns out that this
multi-electrode array system
is actually very
powerful for extracting
many different
cells, the spiking
activity of many
different cells,
out of a piece of tissue.
So what you can do is you put
this through what's called
a "spike-sorting algorithm,"
which uses these different
spike wave forms on these
different electrodes to pull
out a spike train.
And the spike train is now
going to be a delta function
for each different
neuron that you've
identified in this data set.
So even though
different neurons appear
on these different
electrodes, you're
eventually going to extract
this now so that you
have a spike train
for one neuron,
a spike train for
another neuron,
a spike train for a
third neuron, and so on.
And then you can
plot the firing rate
of those different neurons.
This is actually a histogram,
a peristimulus histogram,
of the activity of a
bunch of different neurons
to a movie being played to this
piece of retina in the dish.
And that's just literally a
movie from a forest with trees
swaying around in the breeze.
So you have all these
different neurons.
And you can see that each neuron
responds to a different feature
of that movie.
And that's because each neuron
has a receptive field that's
in a slightly
different location,
has a slightly different spatial
and temporal receptive field,
that it allows it to pick
out different features
of the visual stimulus.
And there are about a million
of these neurons that project
on the back of the retina.
Actually, I should be careful.
It's actually the-- it's
the back of the retina.
Because the light goes
through the ganglion cells
through the photo receptors,
which are actually--
sorry.
Photoreceptors are actually
on the backside of the retina.
Ganglion cells are on
the front, and light
goes through the ganglion cells
to get to the photoreceptors.
And there are million of
those retinal ganglion
cells that then project
up through the optic nerve
to the thalamus.
So how do we figure out
what each of those neurons
is actually responding
to in this movie?
So what we can do is--
you could imagine doing
a spike-triggered average
of these neurons
to the movie that's
playing the trees
swaying in the breeze.
But why would you
not want to do that?
Why would that be a bad idea?
What is it that
we just decided is
the best kind of stimulus to
use to extract receptive fields?
This is a highly
structured stimulus
that's got particular
patterns in the stimulus
and both in space and in time.
So it's really not
an optimal stimulus
for finding the receptive
fields in neurons.
What we want to do is to
make a very noisy stimulus
that we can play, and
that's what they did.
So then they make
this, what they
call in the visual system,
a "random flicker stimulus."
So it's basically a movie
where you randomly choose that
the pixel values, both in R, G,
and B-- red, blue, and green--
for each stimulus
at each time step.
And here's what that looks like.
So now, you play that
movie to the retina,
and you record the spike trains.
So there's the neurons spiking.
And now what you do is you--
because this is now a
two-dimensional stimulus, what
you do is you have to collect
the samples of the movie
at a bunch of time steps
prior to the neurons spiking.
Does that make sense?
Now, you do that for
each spike that occurs.
And now, you average
those all together
to produce a little
movie of what
happened on average before
each spike of the neuron.
And here's what that looks like.
So this is for two
different neurons.
So what is that?
So this is time across the top.
So it starts at
minus half a second.
So what did that look like?
What was it that made
that neurons spike?
What was it that happened right
before that neuron spiked?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Yeah, a dark spot.
So that neuron was excited
by a spot of light,
sorry, by a stimulus that
looked like a darkness right
in that location right there.
So that neuron is essentially
being inhibited by light
at that location.
And when the light at
that location goes away,
boom, the neuron is released
from inhibition and spikes.
Here's another cell.
So that neuron responded
to a spot of light
right there in that location.
And that's because that
neuron gets excitatory input
from bipolar cells
that are located
in the retina at that location.
And those bipolar
cells respond to input
from the photoreceptors
at that location.
That's called an on cell.
That's called an off cell.
So there are many different
kinds of neurons in the retina.
There's something like--
I forget the latest count--
40 or 50 different types of
retinal ganglion cells that
have very specific
responses to visual stimuli.
So now let's break that down
into a spatial and temporal
receptive field.
Most-- I probably
shouldn't say most--
but many retinal ganglion cells
are separable in the sense
that they have a
spatial receptive field
and a temporal receptive
field that are just
a product of each other.
The STRF is a product of a
spatial and temporal component.
So here you can see
as a function of time
before the spike.
So this is the stimulus
at the time of the spike.
This neuron responds
with a spike
to a spot of light that happened
about 150 milliseconds earlier.
And here's what that stimulus
looked like as a function
of space on the retina.
So that's the spatial
receptive field.
Sorry, that's the spatial
temporal receptive field--
a spatial stimulus as
a function of time.
You can write that as a product
of a spatial receptive field
and a temporal receptive field.
So here's what the
spatial receptive field
looks like, and here's the
temporal receptive field.
You can see that this neuron,
just like the example that we
talked about
earlier, this neuron
likes to respond when there's a
darkening in the central area,
followed by a bright spot.
You can see that little bit
of darkening right here.
So the response when this
goes dark and then bright.
So that's the visual system.
Let's take a look at auditory
- you use this same method
for finding receptive fields
in the auditory system.
So we're going to talk briefly.
We're going to come back
to spectral analysis
and spectral processing
of signals in a couple
lectures, but let
me just introduce
some of the basic ideas.
So we're going to
talk about the idea
of a spectral
representation of a sound.
So this is a
microphone signal of--
let me see if you can
guess what it is--
of a creature.
There are parts of this stimulus
that have high frequency.
So this is a microphone signal.
It fluctuates due to
fluctuations in air pressure
when you hear something.
Parts of that signal have
high-frequency fluctuations.
Parts of that signal have
low-frequency fluctuations.
You can compute a
Fourier transform--
which we'll talk
about more later--
as a function of time
stimulus and see what
the spectral components are.
So this is a spectrogram of the
sound that I just-- right now.
But you can see frequency
as a function of time,
and the intensity, or in this
case the darkness on the plot,
shows you how much
energy there is
at a particular frequency
at a particular time,
so frequency as a
function of time.
And now, neurons respond
to stimuli like this.
It's a canary song.
And neurons respond
to different sounds.
And so you can discover
what sounds activate neurons
by doing the same trick.
So I'll show you.
This is from a paper from
Michael Merzenich's lab.
This was worked on by
Christophe deCharms
who was a post-doc
in the Merzenich lab.
And basically, what
you can do is--
OK.
So this is for calculating
a visual receptive field.
For calculating an auditory
receptive field, what
you can do is you can
basically play noisy stimuli
in auditory space.
So what you can do is present
random patterns of tones.
So this is frequency,
and this is time.
And so what you
can do is you can
make a little chords of
tone, tone, tone that last,
let's say, 20 milliseconds.
And then you make a different
random combination of tones,
and then a different random
combination of tones.
And this sounds like a very
scrambled, noisy stimulus.
And you play this to the
animal while you're recording
a neuron in auditory cortex.
The neuron spikes.
And then what you can do is
just do exactly the same trick.
You can look at
the stimulus that
occurred before each spike,
pile up those columns.
There's a little
spectrum temporal pattern
of stimuli that the bird--
in this case, a
monkey heard right
before that neuron spiked.
And you can do the same thing.
You can take that little
snapshot of that sound
and average them together.
And here are the kinds
of things you see.
So that's a spectro-temporal
receptive field.
You can see I plotted it.
It's plotted in a way that
this is the stimulus that
occurs with the spike.
So this is like the D
plotted already flipped.
And you can see that--
how would you describe
what this neuron responds to?
How would you describe that?
So this is frequently.
Sorry, this is
frequency in kilohertz.
And that's time in milliseconds.
So what do you think
this neuron responds to?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: What's that?
AUDIENCE: It
responds [INAUDIBLE]..
MICHALE FEE: It responds
maximally, actually,
to a very short
tone at 4 kilohertz.
You see how it kind of
has some inhibition there?
See how it's kind of
darker right there?
So this neuron actually
will respond better
to a tone that only lasts
20 milliseconds than it will
to a tone that
lasts a long time.
So this is a neuron that
responds to a short tone pulse.
What happens if
we play a stimulus
to this neuron that's broad?
That instead of just
being a tone, [WHISTLE],,
is broad, like [STATIC]?
The noise that's
at 4 kilohertz here
will tend to excite the
neuron, but the noise
that's over here at 5
kilohertz or 3 kilohertz
will tend to inhibit the neuron.
So the best response,
the best stimulus
to make this neuron
respond, is a pure tone
at 4 kilohertz that lasts
about 20-ish milliseconds.
How about this?
Let's take a look at
this neuron right here.
What about that neuron?
What does that neuron
want to respond to?
What does it like to hear?
I'm anthropomorphizing
shamelessly.
You're not supposed to do that.
What kind of stimulus
drives this neuron?
AUDIENCE: [INAUDIBLE]
MICHALE FEE: Good, a
downward sweep, tone sweep,
that goes from about 4
kilohertz to 3 kilohertz.
In how long?
AUDIENCE: 100 [INAUDIBLE].
MICHALE FEE: In about 100
milliseconds, that's right.
Maybe 50 would do,
[WHISTLE],, like that.
How about this?
It's kind of messy, right?
So you can see that neurons
have receptive fields that
can be very complex in
space, or in this case,
in frequency and time.
They are very selective
to particular patterns
in the stimulus.
So we talked about a
mathematical version
of receptive fields,
which are essentially
describing patterns
of sensory inputs
that make a neuron spike.
And we've talked about a
very specific model, called
a linear/nonlinear
model, that describes
how neurons respond to stimuli
or become selective to stimuli.
We've talked about a spatial
receptive field, described
the response of a
neuron as a correlation
between the spatial receptive
field and the stimulus.
Temporal receptive fields,
where we've used convolution
to predict the response of
a neuron to a temporal--
to a stimulus.
We've talked about
the idea of a spatio-
or spectro-temporal
receptive field,
and we've talked about how to
use a spike-triggered average
to extract the spectro-temporal
or spatio-temporal receptive
field of a neuron using white
noise or random stimuli.
