PROFESSOR: Last time we talked
about probability as an
introduction on how to
model uncertainty.
State estimation is one of the
ways that we can deal with
uncertainty in our system.
We can take a model that we
don't completely understand
and attempt to infer information
about it based on
the things that we can observe
about that particular system.
In particular we're going to
look at a set of observations
and actions that either our
system takes or that we
observe about the system and
that we take on that
particular system.
If we continue this process for
multiple time steps then
we can continue to attempt
to learn things about the
particular system.
The process of completing that
behavior over multiple time
steps while making those
inferences is what we refer to
when we talk about
state estimation.
First off, state estimation is
a process that's completed as
a consequence of wanting
to understand a
stochastic state machine.
State estimation itself is not
a stochastic state machine.
State estimation attempts to
take a stochastic state
machine, make a model of that
stochastic state machine, and
then run state estimation on it
iteratively to attempt to
figure out, or recursively, an
attempt to figure out what's
going on inside that stochastic
state machine.
When you build a stochastic
state machine model there are
three components that need
to be specified.
The first is the starting
distribution over states.
For instance, let's say that I
believe that I am sick and I'm
trying to figure out what it
is that I am sick with.
And I could be sick
with three things,
as far as I'm concerned.
I could be sick with strep or
I could be sick with some
other more boring virus.
Or I could be sick with
mononucleosis.
The starting distribution refers
to my starting belief
as to the systems.
And if I'm generically sick in
the general sense, one of the
assumptions that's frequently
made with respect to starting
distributions is that they're
uniform, right?
It could be equally any
of these things.
The second thing you need to
specify when you're talking
about modeling a stochastic
state machine is your
observation distribution.
Or what is the likelihood
associated with making a
particular observation
given that you're
in a current state?
For instance, if I have
mononucleosis how likely would
it be that I observe a bunch of
white ugly patches on the
back of my throat?
Or if I had strep?
What is the likelihood
associated with that?
That kind of thing.
Typically this observation
variable is factored into a
couple different phenomena.
In the sick example
the best thing to
talk about is symptoms.
[INAUDIBLE]
Am I lethargic?
Do I have the white spots on
the back of my throat?
Do I have a fever?
That sort of thing.
The last thing that you need
specify when you're talking
about modeling a stochastic
state machine is your
transition distribution.
You assume that your state
machine is going
to change over time.
Or it is likely that I will
get more or less sick.
And there are things that
I can do to induce
that kind of change.
Or there are things that I can
do that effectively model the
passage of time.
Your actions for a stochastic
state machine model can either
be actions that the model takes
and you were exclusively
doing observations.
But one of the particular
observations that you do also
qualifies as an action or
something that indicates the
passage of time.
Or actions can be something
that you do to
a particular state.
In the sick example, things I
could do to myself to try to
make myself feel better or at
least figure out better what
is going on or what might cause
my distribution to sway
towards one particular state.
I could take antibiotics.
Or sleep in and drink a
lot of orange juice.
Or continue my day as normal.
Given a particular action, any
particular state that you're
starting from, your transition
distribution tells you the
likelihood associated
with being in a
new particular state.
So as a consequence of making
those actions, does the
distribution of likelihood of
a particular illness change?
At this point I'm going to walk
through a step of state
estimation.
Each step of state estimation
is the same.
In fact, if you complete
multiple steps based on the
information that you gained from
the previous step, that's
referred to as recursive
state estimation.
And I'll keep walking through
the sick example.
So when you're doing state
estimation you're trying to
figure out something about
a system that you cannot
perfectly model.
For instance, either your own
immune system or your own
susceptibility to a particular
disease.
And you have all the components
you have for your
stochastic state
machine model.
As a consequence of the passage
of time or as a
consequence of making an
observation and either
observing an action taken by
your stochastic state machine
or performing an action upon
your stochastic state machine
you're going to make a new
estimation of what you believe
the current state of that
unknown system.
Or system that is not completely
observable to you.
You're going to make a new
estimate of your belief of the
state of that system.
In short you're going to solve
for the probability
distribution over
S_(t plus 1).
There are two steps.
The first step is referred
to as the
Bayesian reasoning step.
And it involves performing Bayes
evidence or Bayes rule
upon the current state
distribution given a
particular observation.
So at this point I've
made some sort of
observation about myself.
If I'm talking about the
sick model, right?
I spent all day coughing.
Or I have a fever.
Or my throat is sore.
Or I feel extremely
lethargic, right?
Given that observation I can
take the P(O given S) from my
observation distribution
multiply it by my current
understanding of the
state distribution.
And then divide out by P(O).
The slowest way to complete this
action is to build the
joint distribution and
then condition on
a particular column.
It's very proper.
But you can save yourself some
cycles by doing this.
Let's say I started off
with the uniform
distribution, right?
It could be equally likely that
I have strep or a normal
virus or mono.
As a consequence of making the
observation that I don't have
white spots on the back
of my throat.
I could say, oh the likelihood
of me being in that state--
the likelihood of me just
having a normal virus is
higher and the likelihood of me
having either strep throat
or mono is lower.
This step takes P(S) and
multiplies it by P(O given S).
o Once I have these values I
have to scope back out to the
universe or I have to normalize
these values such
that they sum to 1.
That's where I get my
P(S_t given O).
At this point I've accounted for
the observation that I've
made, but I haven't accounted
for the action on the system.
That's the next step.
We're going to take our results
of Bayesian reasoning
which are sometimes referred
to as B prime S_t.
And take the action and find the
distribution overstates as
a consequence of a single time
step or a single iteration of
state estimation.
The second step is referred
to as a transition update.
We've got our updated belief.
We're going to take our
transition distribution or our
specification for what happens
given that we're in a current
state and an action
has been taken.
At that point we'll have a
probability distribution over
the new states.
And here are my values
from the first step.
As an example let's say that I
sleep in and drink a lot of
orange juice.
As a consequence of sleeping
in and drinking a lot of
orange juice there's some amount
of likelihood that I
will either continue to be
sick with strep or it's
possible that I actually have
just a normal virus.
If I have a normal virus and I
sleep in and drink a lot of
orange juice, this causality
sounds backwards but it's as a
consequence of not being able
to make perfect observations
on the system.
If I have an amount of belief
that says that I think I have
strep and I sleep in and drink
a lot of orange juice, then
it's equally likely that I will
have either strep or a
normal virus after completing
that step, right?
It doesn't differentiate
between the two.
If I'm sick with a virus and I
sleep in and drink a lot of
orange juice, then the state
that I'm going to encourage
myself to be in is
I have a virus.
If I have mono and I sleep in
and drink a lot of orange
juice then there's some
likelihood on the next day
that I will still be
in a state that
looks like I have mono.
But there's also some likelihood
associated with it
that I will be in some state
that looks like I have strep.
That's what happens when you
run the transition update.
When you run the transition
update you end up accumulating
all the probabilities associated
with being in a
particular new state.
As a consequence of being in a
particular previous state and
entering that new state based
on the transition
distribution.
Once you accumulate all these
values you end up with your
new distribution over
a new state.
This represents one step
of state estimation.
If I wanted to run multiple, I
would take the value that I
got here for S_(t plus
1), replace it in
the value for S_t.
And run the same process of
Bayesian reasoning and
transition update.
This concludes my review
of state estimation.
Next time we'll talk
about search.
