[Music]
[Music]
Welcome to this edition of Astrovignettes.
My name is Professor Michael
De Robertis, and I will be your guide on
this journey. Our lesson today will
focus on the scientific method: what it
is, how it is applied, challenges we face
in some of its applications, and
assessing evidence and science's
limitations. It's interesting to note
that the very word 'science' originates in
the Latin word 'scientia', which means
knowledge. By the end of this lesson you
will be able to describe the scientific
method and its five steps, explain the
importance of testability, identify the
different types of evidence and how to
assess them, recognize the significance
of potential bias and how to deal with
it, and understand the limitations of the
scientific method. What, exactly, is the
scientific method? Generally speaking,
science is a method: a method to uncover
truths about the natural world.
It consists of five basic steps. The
first step is to observe a pattern and
to ask a question about it. The second
step is to propose a generalization
based on our observations, and form a
testable hypothesis. Third, we make
predictions based on our hypothesis that
we then test through experiments and
further observations. Our experiments may
or may not verify our predictions. If our
predictions proved to be incorrect, we
have to go back and reject or revise our
hypothesis and its predictions, and then
repeat the testing. If our predictions
proved to be correct, we can temporarily
accept our hypothesis and communicate
its results, usually through peer review.
Let's look at the following narrative
example that illustrates some of these
key steps. When Sara was a child, she
often visited her grandfather's farm. Her
grandfather had some animals on his farm,
including sheep, which Sara liked to
watch. At the farm the sheep all had
white wool. Without realizing it, Sara
came to believe that sheep all have the
same colour of wool. Philosophers refer to
the process of generalizing from a
particular set of observations like this
as
induction. In a way, induction relies on
pattern recognition, which was essential
to the survival and evolution of our
species. Humans rely a great deal on
deductive reasoning in daily life,
including when practicing science. The
belief that all sheep have white wool
might also be called a hypothesis. Later
that year,
Sara's mother bought some wool socks
that were navy blue. Sara was aware that
wool comes from sheep, and so Sara was
puzzled. How could navy blue wool come
from sheep which had white wool? This was
a serious challenge or test to Sara's
hypothesis that all sheep had white wool.
Sara's mother saved Sara's hypothesis
when she explained to Sara about the
dyeing of wool, just like some people dye
their hair. There didn't have to be sheep
with navy blue wool; just sheep whose
wool can be dyed. Next summer, Sara's
grandfather invited her to his farm
again.
She was quite excited, because he told
her he had just bought a few dozen more
sheep. When Sara got to the farm, however,
she was puzzled: among the new flock was
a sheep that had gray wool. Sara knew
her hypothesis was in trouble again. She
ran to her grandfather to ask whether
the sheep with gray wool had had its
wool dyed, but her grandfather assured
her that gray was the natural colour of
the sheep's wool. Sara was in a quandary:
she had to either abandon or revise her
hypothesis. On the basis of this latest
test or observation or evidence,
Sara decided to revise her hypothesis
to 'most sheep have wool that is white,
but a few may have wool that is gray'.
This seemed more consistent with her
observations. Later in life,
Sara would come to realize that sheep
wool, though mostly white in colour, can be
a variety of colours - including gray and
even brown - though never
navy blue. This simple example
illustrates how science actually works.
In practice, each branch of science may
develop its own jargon, and may use
mathematics to a greater or lesser
degree to articulate its hypotheses, but
the scientific method
consists of asking questions or noticing
a pattern; compiling careful observations;
as well as proposing, testing, and
revising hypotheses. A scientist may
spend most of her career on simply
testing and revising hypotheses, relying
on other scientists to acquire data or
frame hypotheses. In the end, however, 
scientists communicate their conclusions
through publications, normally after
careful review by some peers to ensure
sufficient rigour was used to reach these
conclusions. As you have seen, for a
hypothesis to be functional in the
context of the scientific method, it has
to be testable to determine whether it
might be true. There are two important
aspects of testability. First, a
hypothesis must be falsifiable. This
means it must be capable of being shown
to be false. In science there is no
absolute proof for a hypothesis; there is
only a preponderance of evidence. One
might hypothesize, like Bertrand Russell,
that there is a silver teapot orbiting
the Sun somewhere between the orbits of
Earth and Mars. While an easy hypothesis
to state, it would be almost impossible
at our current level of technology to
test. The second concept underlying
testability is repeatability or
reproducibility. If one scientist
concludes something based on an
experiment, that same conclusion should
be reached by another researcher
independently who had performed a
similar experiment if the conclusion
were indeed a fact of nature. In the late
20th century, two researchers claimed to
find evidence for cold fusion: that is,
for a power source in a simple lab
experiment that we know operates in the
centres of stars. If this were true, this
would have amazing consequences for
humankind: a cheap, clean, unlimited energy
supply. Trouble is, scientists at other
labs could not replicate or reproduce
these results, and so this hypothesis
slowly faded away.
[Music]
The quality of evidence is critical in
assessing or testing a hypothesis, and
not just for science but for any truth
claim. Most of us rely on authorities or
authoritative sources to help us assess
certain truth claims. How we choose these
experts is critical if we really are
interested in making an impartial
assessment. Suppose we would like to know
how likely it is that Earth will suffer
a collision with a major Solar System
body in the next century. It would make
sense in this case to consult an
astronomer who specializes in our Solar
System, rather than a website of someone
who promotes government cover-ups and
conspiracy theories. When we become
better informed on the subject, it might
be fun to look over such a website to
identify the techniques used to persuade
the uninformed. But when first setting
out on a search for truth, it's prudent
to steer clear of sources whose
authority is questionable. There are two
basic types of evidence that are
commonly available to humans: anecdotal,
and statistical. Anecdotal evidence
refers to evidence that comes from
personal testimony. Many day-to-day
decisions we make are based on such
evidence. For example, if I really like
the shoes you're wearing, I may ask where
you bought them, so I could get a pair.
Important to me, but pretty unimportant
to society in general. Now what if you
told me that your aunt cured her cancer
by eating potato peels? Should I
recommend eating potato peels to the
next person I learned who has cancer?
Should our government immediately
integrate this therapy into the health
care system? Assessing issues of such
major consequence require much deeper
investigation. This is where statistical
testing comes in. Statistical evidence
and testing allow us to determine
whether there is any real connection
between, say, eating potato peels and
curing cancer, by carrying out a
systematic study. To put this more
formally: is there a correlation between
eating potato peels and curing cancer?
This can be a challenging, time-consuming,
and expensive enterprise, but it is the
only way we
can be sure of determining whether there
is a real connection. It does not make
sense to develop healthcare policy or
deploy limited resources on anything
that has not been carefully scrutinized
in this way. A caveat, however: even though
statistical testing can reveal a
correlation between two variables, it
cannot normally determine whether the
variables are causally related - that is,
what exactly about potato peels cures
cancer. In more formal language, be
careful of making the unwarranted
assumption that correlation implies
causation. Your age is well correlated
with the price of ice cream, but few of
us would say that the two are causally
related. On the other hand, the good
correlation between smoking and lung
cancer is causal. It is also important to
be aware that the more extraordinary the
claim, the more extraordinary the
evidence must be. Should humankind
believe that space aliens are visiting
Earth simply because your cousin saw an
unidentified flying object last week? The
criminal courts follow this strict
requirement: a person is normally not
convicted of murder based solely on
circumstantial evidence. The legal
analogy is also important in
illustrating the importance of the
burden of proof: a person is innocent
until proven guilty. The burden of proof
is on the prosecutor to demonstrate the
defendant is guilty based on evidence.
This is also true in science. The burden
of proof is always on a challenging
rather than the currently accepted
theory. A great deal of evidence has
accumulated over the last century
supporting Einstein's theory of general
relativity, for example, and yet
scientists hear from people all the time
who claim Einstein was wrong and their
theory is right. The burden of proof in
this context is clearly on the maverick
to prove evidence his theory is better
supported than Einstein's, and not on
science to demonstrate the new theory is
incorrect. It is actually a common tactic
of mavericks to demand that scientists
must prove them wrong. It is also helpful
to be aware that it is very difficult to
prove a negative. While it may be
possible to provide excellent evidence
that a live, full-grown elephant is not in the 
room with you at this moment, it is
nearly impossible to prove something
more general, like there is no tooth
fairy. Humans, including scientists, are
susceptible to a variety of challenges
when working with data. In science,
observations used to test the hypothesis
can be biased: that is, they do not
faithfully represent the objects or
populations under study. We distinguish
between three major types of biases:
technical or analytical bias, cognitive
bias, and confirmation bias. Technical or
analytical bias can be a result of using
instruments that are flawed, or because
the observational techniques are flawed.
For example, you wouldn't weigh yourself
on a scale that registers twenty
kilograms before you even stepped on the
scale. Or, suppose you were asked to
consider how fast the average twenty-
year-old female can run a distance of
200 meters. It would be foolish to use
data from the finals of this event at
the last Olympics to make such an
estimate. Human beings are subject to all
sorts of cognitive biases. We evolved
over very long time-scales; our very
survival was aided by the ability to
identify and interpret patterns - patterns
that may not even be there. A rustle in
the bushes may well only be the result
of a breeze and not of a tiger ready to
pounce, but we couldn't very well take
the time to perform a scientific
experiment on the spot to see which
hypothesis is true, could we?
In a similar manner, our brains often
identify two sequential events as being
causally related, even though they aren't.
For example, suppose you saw a black cat
walk across your path on the street, and
then you got into an argument with your
best friend later that day. Superstitions
sometimes emerge from this type of
erroneous thinking - for example, that a
black cat brings bad luck. One of the
most common biases we encounter is
called confirmational bias. For example,
you really hate checking out at a large
grocery store because you swear you
almost always seem to get in the slowest
line. Maybe - but you're less likely to
remember occasions when your checkout
time was average, or even quicker than
average, and more likely to recall the
few annoying times when you chose a slow
line.
The latter nicely confirms what you
already imagined to be the case, whether
true or not. This type of bias is very
common in science and in everyday life,
contributing to some forms of prejudice.
It is also a significant contributor to
the increasing polarization we
experience in the current era of fake
news. We find news told from a
perspective more sympathetic to our own
view is more trustworthy than news told
from a challenging perspective. Ideally,
we should assess the value of each item
on its own merit.
[Music]
[Music]
There is no doubt that the scientific
method is the most effective way of
determining truths about our natural
world - the world of atoms, bacteria, stars - 
but it has limitations that need to be
respected.
First, the scientific method deals with
propositions involving the natural world;
proposition connected by reason and
logic.
It makes no prior assumptions about the
supernatural, or even whether the
supernatural world exists at all. As a
result, science cannot affirm whether
there is or is not a G_d, for example.
Second, some people, particularly buoyed
by the success of science over the past
few centuries, maintained that the
scientific method is the only reliable
method for determining truth, period.
Quite apart from reducing other areas of
human knowledge, such as the study of
history or literature, to lesser
importance, it denies the existence of
the supernatural by definition. This
inappropriate extension of the
scientific method
has recently become more common,
including among scientists, and needs to
be identified for what it is. Third, and
finally, science is often called upon to
make moral or value judgments. It is
important to note that science itself
cannot make value judgments; people can
make value judgments based, in part, on
scientific information, but humans, not
science decide whether something is good
or bad. Let's review what we have learned
from this vignette: the scientific method
is a highly reliable way of determining
truths about the natural world. It is
composed of five steps: observations and
asking questions, making generalizations
to form a testable hypothesis, making and
testing predictions based on the
hypothesis, and then either revising or
rejecting the original hypothesis or, if
it's verified, communicating the results.
For a hypothesis to be functional in the
context of the scientific method, it has
to be testable: that is, both falsifiable
and reproducible.
When assessing evidence, one must be
aware of the role of authority. We also
need to distinguish between anecdotal
evidence versus statistical evidence and
testing. One of the two basic tenets of
assessing evidence is that the more
extraordinary the claim, the more
extraordinary the evidence must be. The
second tenet is that the onus is always
on the challenger to demonstrate the
superiority of their hypothesis over a
currently accepted hypothesis. The
application of the scientific method is
subject to human error;
more specifically, bias. Three of the most
common biases are technical or analytical
bias, cognitive biases, and confirmation
bias.
[Music]
[Music]
[Music]
[Music]
Much like all other human endeavours, the
application of the scientific method has
limitations. Those who engage in the
application of the scientific method,
whether they be scientists or non
scientists, must always be acutely aware
of these limitations. Thank you for
taking this journey with me on the
scientific method.
[Music]
[Music]
[Music]
[Music]
[Music]
[Music]
[Construction noises]
[Whoosh]
