Welcome back. In this third and final
session on philosophy of science, I'm going
to take up the question of theory choice
and how do we know that one theory or
one explanation is better than another
theory or another explanation. So, this
question has a long history. the
philosopher Karl Popper argued that a
critical feature of theories that we
should make them falsifiable, we should
make theories that have falsifiable
observable implications and that it's
the failure to falsify theories that
constitute some form of progress, and
this is often referred to as the
falsification as an approach to thinking
about progress and choice among theories.
The problem with this view, or one of the
problems--a key problem--is that when a
theory fails in making the prediction or
explaining a case, it's not always clear
if it's the theory that failed or maybe
some weird thing was going on, or maybe
there was some measurement error, or
something else. this is known as the
Quine-Duheim problem, after two different
philosophers of science. And so it's not
always clear that we should throw out a
theory at the first problem or anomaly
and indeed when we look at the history
of science, scientists don't usually
throw out long-established theories when
they don't fit one experiment to one
case. They usually look first for some
flaws in the measurement or flaws in the
experiment. so Thomas Kuhn looked at the
history of science, and extended this
argument further, and said really
competing theories aren't fully
commensurable--they don't really fully
address the same set of issues--and so
it's not obvious, you know, when theories
fail to explain some problem means
that the other theory is necessarily
better. And so, he famously argued that
there's no higher standard for theory
choice than the agreement of what he
called the relevant scientific community.
Now there are many critics of his view
because this seemed to throw all of
science just into whatever scientist
thought was right or thought it was.
It's way too relativistic and
sociological for for many people's
tastes about what they think that
science is. At the same time, it's more
neglected in Kuhn's work, but well he
still said that, you know, it's the
relevant, the assent of the relevant
scientific community that matters. He
also said that it makes a great deal of
sense to ask which of two theories fits
the facts better. So it's hard to
reconcile those two views in Kuhn. The
one seems very empiricist, and the other
is very sociological notion of whether
and how science progresses. So, Imre
Lakatos, who was sort of a part-time
philosopher of science, I believe he is a
mathematician, mostly. He was very
uncomfortable with Kuhn's argument. He
thought it made science way too
relativistic, and, and soft. But he also
was wary of what he called "naive
falsificationism," the Popperian idea
that you sort of throw out a theory at the
first anomaly because clearly that
wasn't empirically true in it and Kuhn,
no Lakatos wasn't convinced that it is 
logically justified to throw out
theories at the first anomaly. So he
wanted to find a way to assess whether
theories were progressive and improving
or not that was rigorous and not too
dependent on just the the willy-nilly
views of a few different scientists. So
he wanted to find methodological
criteria for a more sophisticated form
of judging theoretical progress. So he
argued that the choice between comparing
theories or explanations is not made by
simply falsification or one test or one
anomaly, but it's a series of tests over
time and a series of anomalies or
growing anomalies that lead us to move
away from one theory and towards some
alternative. And we don't also--also we
don't throw out a theory unless there is
an alternative. Scientific progress is a
three-cornered fight between two
theories, at least two theories, and the
evidence. If there's only one theory, it's--
it's the best theory in town until
another different theory comes along. So
he also argued that choice between
theories proceeds not just by
falsifications but by confirmations of
their predictions and expectations. Now
he said that the choice between theories
should be made on the basis of what he
called "novel facts." Now he didn't--he
didn't define novel facts as tightly as
we might like, but this was the idea that
theories would lead to a series of some
facts that were, in some sense, novel or
new and that those theories would be
judged progressive and the ones that
face growing anomalies would be judged
regressive. But later people, in
particular, Colin and Miriam Elman have
come up with a more clear discussion of
different kinds of novel facts, and there
are two kinds, in particular, that we want
to focus on here. One is what we call
"background theory novelty." This is the
idea that if a theory says it explains
some outcome or some process and no
other theory says it has an explanation
for that, well then the first theory by
definition it's the best theory, right? We--
we sometimes talk about inference to the
best explanation and when there's only
one candidate explanation, well, that's
the best explanation in town. So an
example here would be when Einstein came
up with his theory of relativity, there
was something that was already known to
be an anomaly for Newton's theory, which
was that the orbit of mercury
around the Sun did not fit the
predictions of Newton's theory. So
Einstein knew this already, but it turned
out that the mathematics of Einstein's
theory fit the orbit of mercury. And so
this was taken as a strong support for
Einstein's theory because it was the
only theory that had some plausible
argument that it was consistent with the
orbit of mercury, right? So, this is
background theory novelty. The other kind
of novelty is what we call use novelty.
Use novelty is when we have a theory
that explains some facts or outcomes
that were not already used to create the
theory in the first place. This could be
a prediction about event that hasn't
happened yet,
or it could be a prediction about
some evidence that hasn't been looked at
even if the evidence is in the past.
So, to take another example from
Einstein's work, Einstein's theory of
relativity suggested that if we could
measure the light of distant stars as
that light traveled close to the Sun, we
would see the gravitational field of the
Sun actually pulling that light towards
the Sun. Again, this was not consistent
with Newton's Newton's theory so it had
background theory novelty as well as use
novelty. It had used novelty because
nobody had ever before measured whether
light bends toward the Sun. Indeed, the
only way you could measure that is
during an eclipse because otherwise the
light of the Sun you know is too bright
for you to see the light of distant star.
So, in the solar eclipse of 1916
scientists at different points on the, on
the earth took photographs of the light
of distant stars as that light went
close to the Sun, and indeed those
photographs suggested that the
measurements were more consistent with
Einstein's theory. And so, this was also
taken as a strong confirmation of
Einstein's theory because it was
something that was not known for certain
and had never been observed before. And
to be consistent with that was quite a
strong demonstration in support, sort of
confirming the theory. So you will often
hear that you can't create a theory from
a case and test it against the same case
and the idea behind that is, is that or
the idea that leads to that is some
emphasis on this idea of use novelty. But
I would say that's wrong. I would say you
can develop a theory from a case and
test it against different evidence in
the same case because that evidence
still may not have been examined before,
you may not know
with the value of that evidence before
you observe it, right? Now we'll see when
we talk about Bayesian process tracing
that some Bayesian process tracers,
notably Tasha Fairfield and Andrew
Charman, think that actually it doesn't
matter if you knew the evidence already
when you came up with a theory. It just
matters whether that evidence is likely
if the theory is true and whether that
evidence is likely if alternative
explanations are true. We'll come back to
that argument and explain it. Now, just a
preview a little bit more, I would say
that a lot of people have the intuition
that there is a difference between
evidence that you already knew what it
was and evidence that you didn't know
what it was and I think there's a
logical and there's a psychological
explanation for that intuition. The
logical argument for it is that if a
theory was built around evidence we
already knew, then that evidence can't be
inconsistent with the theory, can't test
the theory, right? In Bayesian factual
state, we'll see there is an argument
you're still, in some sense, testing the
theory. But the psychological argument is
that because of all the ways we're prone
to confirmation bias that if we know
about the value of evidence already,
there's subtle ways that we'll sort of
treat that as if it's a confirmation of
the theory when really we may be just,
you know, fitting the theory around the
evidence or what sometimes we call curve
fitting, right? So my argument, which is a
little bit different from Fairfield and
Charman, again, we'll come back to this, is
that probably there is still some
psychological difference between
evidence we already knew what the
evidence was versus evidence that we
only discovered after, after we thought
to look for it, and after we'd already
specified how likely was that evidence
if this theory is true or if an
alternative explanation is true. We'll
come back to that issue. But I just
wanted to preview it here. Okay,
now the reason that I emphasized these
two kinds of novelty, and especially the
second kind, is that in case study
research we are often trying to test
different explanations against evidence
in a case and there's going to be lots
of evidence in a case that we don't know
beforehand what the
that evidence will show. There's going to
be lots of details that even people who
were participating the case, even people
have studied closely may not know all
the details of the processes in that
case. And so, there may be lots of
evidence in the case that can be still
independent in the use novelty sense,
right? So, I would say, I will come back to my
statement that often we can develop a
theory or an explanation from evidence
in the case and then test it against
different and independent evidence in
the same case, right? Or again, we could
even make an argument that we could
still test it against the evidence that
gave rise to the theory in the first
place, if that evidence is unlikely to be
true under the alternative explanations.
That's the sort of objectivist Bayesian
view versus the more subjectivist
Bayesian view, which still puts some
value on use novelty. We'll come back to
that when we talk about Bayesian process
tracing. Okay, let me close them with just
coming back to some of the postmodern
critiques of progress and of theory
choice and you might, at the end of the
video, if you're taking a course with me,
you might write down some notes or
questions you have on this or any other
parts of this presentation. So, one
critique is that observation is theory-
laden, right? We have theories in our heads
that literally sort of shape what we can
see and perceive in the world. And
there's lots of psychology experiments
that show that that's true. And it is
true, right? At the same time, I think the
short counter argument is that
observation is not theory-determined.
There will still be things in the world
that will surprise us given our theories
and we will experience that feeling of
surprise and we can use that to explore
something that doesn't fit with our
theoretical expectations, right? So we--there, there are some aspects
of social relations in the world that
are independent of us as observers, and
we can recognize those things because
they're--they're different from what our
mind led us to expect, right?
The second critique of
of theory testing and ideas of progress
is that theories and language are open
to many different interpretations and I
would say, again, yes that's true, but even
people who make those arguments, often,
act in their own lives in their own
personal lives and in their scientific
or productive lives, as if some
explanations are better than others,
right?
And they may have some ideas for judging
when some interpretations, or if you will,
explanations, are better than others. And
so I would say we do have some ways to
judge this issue, not perfectly, right? But
we can make some claims--use novelty
and background novelty, or, or two
standards I've given you, for judging
progress in theories. We will see that the
Bayesian logic also gives us a way to
judge alternative explanations. A third
critique is that theories and science
reflect power and social power as well
as evidence. And that's, again, that's true,
right? So, some kinds of research get
funded, other kinds of research doesn't,
and powerful wealthy actors may have a
lot of influence over what gets studied
and what doesn't. At the same time, my
response to that would be we know that
and we can still study that. Even when we
don't have a lot of funding, we can study
relations of power. And sometimes we will
get funding to study relations of power
and deep relations of power are one of
the most studied things in all of social
science. So yes, we can be conscious of
that and we can try to correct for that
as best we can. We're not totally
helpless here. And then the final
critique is that agents and structures
may be mutually constitutive--maybe
endogenous all the way down to even
small slices of space and time. Again,
that can be true, right? But I would still
argue that social structures are
sufficiently long-lived
that we can usefully study them and we
can usefully make claims about them and
we can hopefully influence the progress
of society for the better for,
for all individuals, right? Notice that
there's a kind of attention between
claiming that, you know, science is so
infused by social power that it--it can't
cast a light on social power and saying,
you know, everything is up for grabs all
the time, agents and structures are so
mutually constitutive that sort of
anything could happen.
Well, those can't both be true. If we have
structures of power in society and we do,
they will be lasting--they will reproduce
themselves over time and we can study
them. Maybe not, we can't make claims for
all time because institutions change,
right? Even the biological nature, humans
have changed--evolved over time, right? But
we can make claims for pretty long
periods of time because institutions
and human biology don't change that fast.
And so I don't aspire to make
predictions or explanations for all of
time but for very meaningful periods of
time. Okay, let me end with some
implications for case studies. Process
tracing relates to the study of causal
mechanisms and how they operate in
individual cases. We'll unpack that a lot
more when we talk about process tracing.
This uses, I would argue, Bayesian logic.
We'll say a lot more about that down the
road. Some advice about how to do
research that emerges out of a
statistical approach and statistical
mindset does not apply to case studies.
So advice like do not study single cases
does not apply. There's a lot you can
learn from single case studies. Claims
like do not select on the dependent
variable, often for very good reason we
select partly on our knowledge of the
dependent variable in case studies. There
are reasons you wouldn't want to do
those things in statistics and they're
very good methodological reasons. They
don't apply to case studies or at least
not in the same way, right? So these are
not absolute rules for case studies,
right? And I would say these criteria--
background novelty and use novelty--are
useful in case studies. They help
discipline us from succumbing to
confirmation bias, and they're useful in
carrying out process tracing tests,
although we'll see there's some
disagreement even among Bayesian process
tracers of whom I consider myself one
about whether use novelty is really a
useful or essential criteria. So I think
finally we need to address the critiques
made by post modernists, but we shouldn't
be paralyzed by them. They don't--they
don't take away all the prospects for
progress or science. And indeed, when you
read a lot of interpreters like Clifford
Geertz and others there, you may use
different language, but I would say
they're actually making arguments that
some explanations are better than others
and they're ways to assess that, and to
test that and that there are some ideas
of progress in our understanding of the
social world. Okay, let me stop there, if
you're taking a class with me, take a few
minutes and write down any questions you
have. That we can take up when we
interact with each other on whether it's
in person or on Zoom.
