Back in the late '90s we introduced the Baloney
Detection Kit, inspired by Carl Sagan’s
'Demon-Haunted World' where he had a chapter
on the Baloney Detection Kit.
He had his set of questions; I kind of developed
my own because I started encountering other
people that disagreed with me, you know, “we
never went to the moon” people, conspiracy
people, whatever, and I thought okay so: How
do we know—if I don’t know what’s coming
down the pike in ten years from now, if I
am going to teach my students how to think
critically, what are the key points, like
just basic questions they can ask?
So it begins with one: how reliable is the
source of the claim?
Here’s the claim, how reliable is it?
What’s the evidence for it?
What’s the quality of the evidence?
Where does it come from?
Who said that?
Is this some fake news alternative site thing
or is it the Wall Street Journal or The New
York Times?
I mean, the source really matters.
Has anyone tried to disprove the claim?
This is super important because everybody
thinks they’re right and every website has
testimonials about this product or that idea;
the question is not what do your supporters
think but what do the people who don’t agree
with you think?
Because that’s what I want to know.
Has anyone run experiments to try to disprove
your theory?
In science, this is as basic as it gets.
Karl Popper called this the principle of falsification,
that is we can’t ever prove a theory correct,
but we can disprove it by having an experiment
that shows it’s wrong.
If you can’t falsify it, what are you really
doing?
And my favorite story on this, by the way—let
me just have a little sidetrack here from
Carl Sagan, he’s got this great little section
in his book 'Demon-Haunted World': “There
is a dragon in my garage.
I have a dragon in my garage.
Do you want to see it?
Let me show you.”
So I pull up the garage door I go, “Look.
Can you see the dragon?”
And you look in there and you go, “I don’t
see anything.”
“Oh, sorry, this is an invisible dragon.”
“An invisible dragon?”
“Yeah, yeah he’s invisible.”
“Well, what if we put some flour on the
ground and then we’ll get the footprints
of the dragon.”
“Well, no, see, this is a special dragon
that hovers above the ground, it floats.
It’s an invisible floating dragon.”
“An invisible floating dragon.
Okay.
Wait, I have some infrared cameras here we
can detect the heat of the dragon.”
”No, see this is a cold-blooded dragon.
It doesn’t give off any heat.”
“What about the fire?
We can detect the fire that the dragon spits
out.”
“No, it spits out cold fire.”
You see the problem?
If there’s no way for me to falsify that
there’s a dragon there, what’s the difference
between an invisible floating heatless dragon
and no dragon at all?
None.
And of course we can apply this to god or
any other supernatural/paranormal-type phenomenon.
If I can’t debunk it, if I can’t falsify
it, if there’s no way to test it, then how
will we ever know it’s true?
This is the core of the Baloney Detection
Kit.
We have to be able to get to whether it’s
true or not in some way, so it’s not just
my opinion versus your opinion and we shout
at each other.
Then we want to know: does the claimant’s
personal belief somehow enter in?
Because of course we all have personal opinions
and beliefs about things; my politics, my
religion, my ideology can influence me.
It doesn’t make it wrong, but it’s good
to know if somebody has an agenda.
So when you watch Fox News you know that they
have an agenda for sure, or there are other
sources on the left that have a liberal agenda,
maybe NPR, who knows.
But you see it’s good to know that just
in case, so when you hear the fact you go,
“Well maybe, but I know this guy has an
agenda.”
So that’s the kind of thing.
Does the new idea being proposed account for
the same amount of information that the old
idea does and some of the new anomalies that
the old idea can’t explain?
So people offer theories, so-called alternative
theories of physics, for example, and they
always send them to me going, “Hey listen,
I’m not good at math but if you help me
with the math I’ll share the Nobel Prize
with you.”
…Right.
I don’t do math and physics so you might
take it to the local high school physics teacher
before you announce that you’ve made the
greatest discovery since Newton and Einstein.
The problem with that is: the current theories
do pretty well at explaining most everything.
Not everything, so there are always anomalies
we can’t explain so maybe there’s some
new theory coming down the pike that might
account for all the old theories, explanations,
and the new anomalies that the old theory
can’t explain.
Maybe.
But again, we’ve got to be able to test
it first.
So those are the kinds of things.
Does the claimant play by the rules of science,
the rules of the field that you’re in, for
example?
Again, these alternative physics guys come
to me.
Don’t come to me, I’m not a physicist!
Did you at least ask the local high school
physics teacher if this makes any sense?
Because if you have no training, you don’t
know all the mistakes that people in the past
have already made to get to where they are
now, and if you’re starting here without
that background, you’re going to make lots
of mistakes.
So these are the sorts of things that any
good baloney detector should know.
I call this Skepticism 101, it's the course
I teach at Chapman University to incoming
freshmen; it’s a critical thinking course,
but it’s really just how to detect baloney.
And not everything is baloney, some things
turn out to be true like the theory of evolution,
the theory of the Big Bang, germ theory of
disease, plate tectonics in geology—these
are things that were once radically heretical,
and now they’re accepted.
How did that happen?
It happened because they have evidence, and
that’s what you need.
