[MUSIC]
My name is Rosanna Perotti. I'm a
professor in the political science
department here and I want to welcome
you to this talk, to students and friends.
Political Science Talks Politics. This is
the third installation, Polling 101, and
we have Dr. Craig Burnett, associate
professor and Kalikow School Poll Program
director, delivering this talk. Before we
get started, I want to say first that
this is the third in this small series
of talks this semester. The series will
continue next semester with talks about
various aspects of politics. I want to
say at the outset, thank you to the folks
who have helped to put this on from
University Relations and the Hofstra
Cultural Center to Professor Meena Bose
who and the Peter S Kalikow Center for
the Study of the American Presidency for
co-sponsoring this program introducing
dr. Burnett he has been a professor in
our department for a very short time and
in that very short time he has given so
much of himself and we appreciate so
much the expertise that he's added to
our our faculty he teaches courses in
state and local politics and also in
political methodology and for years he's
been interested in an active in polling
and the measurement of public opinion is
one of his major interests as is of the
study of democracy direct democracy so
let me hand the podium to him dr. Craig
Burnett
thank you dr. parati and doubling the
thanks to everybody who's who's helped
make this happen when this idea came
about there was a sense that you know
there might be a little bit of a gap in
between you know what the public's
knowledge is about polling yet it's fair
to say i think that we're inundated with
polls you know every day I can refresh
various websites to see anywhere between
8 and 10 polls have released information
and this is this is my attempt to kind
of help everyone else make sense out of
this I love this stuff I follow it
closely but you know there are some sort
of best practices and some tools that I
think everybody can use when it comes to
looking and understanding exactly what
they're looking at so that's what that's
what this talk is about and we will go
through a lot of topics because polling
is complicated it's changing and
probably changing even more in the last
10 years than I think has ever done in
probably any 10 year period before so I
can't tell you exactly what I think the
next 10 years is gonna look like but
it'll certainly be a lot different from
now so yeah I tend to practice what I
preach and I always tell people you have
about a minute of their attention before
they start doing something else whether
it's playing on their phones or it's you
know just drifting off into space and
thinking about something else so if you
don't remember anything else from this
talk here's what I want you to know
polls are snapshot in time this means
that they are for the time period in
which they are taken the best
representation we can get it does not
mean they were valid for anything that
happened before or anything that's gonna
come in the future
second polls are still valid and they're
still valuable there's been some
discussion about this and we'll talk
about this as we get into the
nitty-gritty but they are in fact still
useful and so the discussion I think
that has been coming from various places
in the public about whether or not
they're still useful I think is is
misguided and then finally
polling will continue to face challenges
as it evolves
or time I think this is a given I think
the challenges are unique to this
particular time in the way the society
is changing I do think that we'll figure
it out but it is in fact going to be a
significant challenge so here's the
general outline of the talk we'll talk
about really the sort of start of it all
how did we get to modern public opinion
research you know but what sort of
brought this about then I'll take you
through a very brief discussion of how
public opinion and research has evolved
over time we'll talk about how to
construct a poll then we'll talk about
how to read a poll we'll talk about
trends and then we'll talk about what
are the challenges that are facing
public opinion research
well then we'll throw in some fun sort
of things is like where else can you get
measures of political opinion and so
I'll show you a betting market and what
that's about
and then we'll conclude with a
discussion of can I trust public opinion
polls so whenever we talk about modern
public opinion research most people will
start with the story of literary digest
and so literary digests for those of you
who are not familiar with it was for its
time one of the most widely read news
magazines in the country and in 1927
they had a circulation of over 1 million
the way to think about this is that
literary digest is sort of similar to
Time magazine wise more near its height
of power it was well known for lots of
things it was sort of amalgam of
interest but one of the things that
became extremely well known for is a
straw poll and what it would do is it
would send out these sorts of cards you
can see here on the bottom right here
this is what they would send out to
people they would send it out to their
readers and they would send it out to
addresses that were associated with
other readership lists and they were
hoping that they would be able to get
people to respond and tell them how they
think they're gonna vote and then they
would use that information to make a
prediction about the the upcoming
election turns out it was it was
generally okay most of time but there
are inherent problems with straw polls
right they're not necessarily
representative and this is this is going
to be their problem in ultimate downfall
in 1936 they get to about ten million
people so this is not some small amount
of people that they've contacted this is
actually a huge number right they get
2.4 million people to respond
awesome sample size you would say to
yourself if I could get two point four
million responses today boy what could I
do with that data so this sounds by the
numbers pretty impressive but it was
anything but perfect so this is the poll
that they ran in 1936 and they come up
with a formula you know just adding
together and seeing exactly who's gonna
win why wouldn't they
you remember president Landon don't you
clearly he's just somebody who's been
thrown aside to the dustbin of history
no no there was no president landed but
they thought there would be in 1936 and
they found that to be a margin of
millions of votes in fact and this is
the response the week after the election
they'd find out that they were wrong and
they weren't just from they were
significantly wrong by a big margin so
this actually is the beginning of the
end of literary digest this diss
discredit them so widely that they
actually just go out of business shortly
thereafter also during the 1936 we see
George Gallup first introduced and this
is sort of starting for him it's kind of
a hobby he wasn't really traditionally
interested in public opinion but he also
runs a poll in 1936 and he only uses
50,000 respondents but he does it in a
different way right he goes to each
state and he tries to gather a
representative sample from each of the
states because he understands unlike
what literary digest did not is that in
fact it's not just sheer numbers that
matters but it's who's voting and where
and so with 50,000 he actually predicts
pretty clearly that Roosevelt's going to
win the interesting thing that he also
does is that he correctly predicts the
literary digest results because he
mimics their sampling method so in both
cases he was right
Roosevelt won the actual election and he
was able to successfully predict that
literary digest would get it wrong so
what went wrong
straw polls are not scientific so we
don't really do straw polls in a
significant way anymore probably the
most significant one that any major news
headline would would grab with me maybe
the Iowa straw poll and that's only
because they're the first state but you
know coming with that is everybody
understands it's not scientific the
problem particularly with literary
digest was that they had what's called
non-response bias and that was the
response rate was only twenty four
percent so most the us almost a lion's
share of their surveys did not get a
response second is they have a selection
bias this is because this is 1936 so
we're at the height of the depression
one of the first things that goes away
when it's time to cut your budget
because there's a depression on are
things you don't need which would happen
to be subscriptions to literary digest
and and subscriptions in general so
these sorts of extraneous items and
they're polling lists and their contact
lists were were generally biased toward
people who had money who were doing
reasonably well during the during the
Depression and I don't want to say that
that means that we should then crown
Gallup the king and you know never think
that they can ever do anything wrong
because they clearly had in 1948 they
actually did predict that Dewey was
going to defeat to him in as well Gallup
says that the reason he was so wrong as
he stopped polling a couple weeks before
the election so everybody gets it wrong
some of the time is the lesson to take
away from this so now they'll talk a
little bit about a brief history of
where we've gone with methodology the
first method that really kind of
everybody was doing at the beginning of
sort of academic research or in the case
of Gallup or literary digest was either
in person or mail based surveys these
are necessity these are expensive
methods right because they require you'd
either use the mail service
or in cases of in person somebody to
travel you have to put them up
transportation cost food etc extremely
expensive and they have to be carefully
done the second that sort of changed the
way that we fundamentally do public
opinion research is through the
telephone right this was in 1936 not yet
quiet in every household it wasn't
necessarily understood that you could do
these sorts of things but what happens
is the advent of sort of the telephone
is belonging it is now has a place and
everybody's household you can randomly
dial numbers and get a random house and
if you do this enough times you end up
with a nice representative sample
everybody has the same probability to
end up in the sample you don't have to
worry about things like cluster sampling
and this is sort of revolutionary
because it's a lot cheaper right you
don't have to do mail you just you just
get on the phone you make a phone call
sure you might have some sort of things
like you know long-distance fees this is
gonna be a stretch of the imagination
for therefore the young people in the
audience but in fact this was a cheaper
way to do things and it worked and it
worked really well and it totally
revolutionized the way that we do it the
problem was is that nobody saw the cell
phone coming right we all knew that we
liked them when we first got our hands
on them but the fact of the matter is
nobody ever predicted that we would ever
give up and give up our landlines and
this has created a problem for public
opinion research that you cannot rely
upon random digit dialing anymore just
by a show of hands how many you guys
have a landline that is not your parents
okay we got a couple I see four five six
and we got a full house here so what is
this 120 people or so so I mean you know
you do the math but it's not good
alright so those are the people who are
eligible to even be called let's not
even get into the fact that if you get a
call on your landline what's the
operating assumption that's a spam right
yeah so you wouldn't answer it anyway so
that gets us to the future and where
we're sort of on to next right and
that's the online format and this has
own set of challenges and has its own
set of biases but it's extremely cheap
and more and more and more it is the
case that people have access to online
formats whether it's through your phone
whether it's through your computer so
this is where the next frontier of
public opinion research is going to be
and it's in many ways it's already here
and so that's not to say that some you
know that there's still a value in doing
telephone surveys there certainly is
it's also a value to doing in-person
interviews or male based surveys - they
all have their advantages and they all
have their challenges so let's talk
about how we put a poll together
the first thing that we would do is
define our population who are we trying
to get after who are we trying to say
something about right that can be a lot
of different things it could be the
entire world is depicted here could be
adults in America could be children in
America could be voters which is
typically what we're most likely after
like we're typically after when we talk
about public opinion polls we're after
likely voters not just people who are
registered right even though we want to
know what they think - but we're
actually interested in the people who we
think are going to show up on Election
Day right those are the ones that are
going to be casting their ballots
ultimately those are the ones that are
going to be deciding who the winner is
so this is what most polls are after
especially polls that are trying to
predict outcomes then we create a
questionnaire right so this is where you
would sit back and you would think to
yourself what is the point of this
survey is this trying to predict the
outcome of an election is it more
focused on policy is it a mix of both
and then you build out your
questionnaire based on that right that
is how you would think about you first
set the goal you set a parameter about
what you hope to be able to say
something about and it should be focused
right you shouldn't just ask about
everything because you can chances are
you want to be able to say something
very specifically but I do want to add
at this point which is question wording
is an art it's not a science and any
time you write a question chances are
you are going to build in to that some
sort of bias or some sort of push or
some sort of poll that is going to cause
answers to change that's just the truth
there is no predefined way to do this so
when you read a poll from somebody even
like the New York Times or CBS those
questions are designed by people and
those people carry with them all of the
thoughts all of their history and all
the biases that they have with them so
the goal of a of a pollster at least one
that's trying to do their job well is to
in fact write a question as cleanly as
possible with as least bias as possible
but it is never a science I cannot tell
you that any question ever asked is a
perfect question there's just no such
thing then after you've done all that
you set your population parameters you
set up your questionnaire then you draw
a sample right and so there are many
ways to do this it depends on which
method you're doing could be a mixed
method approach could be you're calling
all landlines could be some landline
some cell phones could be that you're
calling a mix of cell phones landlines
and also may be doing an online
component with that that seems to be
more and more at least the way the
academic research is going but it's a
sample right it's like dropping a bucket
into the well you're not gonna get all
of the water out but you're gonna get
some of the water out and you're hoping
that that water looks like the rest of
the water that's down there and chances
are it well right if you're doing your
if you're doing your job well and your
sampling scheme is drawing as good of a
representative sample as possible that
should be true that doesn't mean that
you can't possibly pull a bad sample
certainly it happens and it'll happen
again and it'll happen to even the best
pollster so it's just the fact of life
but if you're doing it as best you can
this should be true so in telephone
research you're generally going to go
down to and I'm focusing on these
because these are the two most
predominant methods out there random
digit dialing of landlines cell phones
and I should be clear here you can still
random digit dial landlines you are not
allowed to randomly digit dial cell
phones so that means that somebody who's
doing a cell phone based survey or at
least part of it means that somebody is
operating that on the other end of the
line who's punching that number in
physically punching it instead of having
a computer do it it's a very precise
definition law that keeps them from
doing random digit dialing of cell
phones so if you're wondering why you
haven't been surveyed by a major polling
organization you only have a cell phone
that could be why it typically means
they have to buy a list of cell phones
and then choose from that list so maybe
you're not on that list yet but you
probably will be soon and then finally
for online there's two methods there is
a probability based and then an on
probability based methodology
probability based is still probably
considered the gold standard although
it's a little bit more difficult to do
why because we know that some
populations are less likely to be online
than others and so you have to do
something about that so probability
based polling outfits of which there are
a couple I know for example if so studs
one I just was looking and I think Pew
also does some of this as well they must
incentivize the ability to be online to
do this so perhaps they're paying for
their internet or in some cases actually
providing the instrument like a computer
to do that so it's becoming less and
less of a problem because cell phone
adoption is so wide in today's society
but the fact of the matter is there's
still some people who are less likely to
do this probability based means that you
have at least the saint you have the
same exact percentage of possibility of
making the sample that's a more
expensive way to do things because now
you have to be able to define who those
probability who that is be able to
randomly select from that that's
difficult what we see more and more are
the non probability based samples and
they're not necessarily bad in fact many
of them are extremely sophisticated but
it means that you did not by definition
have the same probability of ending up
in the sample as somebody
in a probability-based and we'll dive
into this a little bit more but once
you've done that once you've gathered
your sample that's when you do the
analysis right you analyze the data you
report the results and then you think
about how you want to publicize them if
you're a research arm of some sort of
organization chances are you're going to
publish this if you're pew you're gonna
put it on your website you're gonna make
a lot of noise about it but even then
even if you have an in-house publication
strategy
you're still gonna want to reach out to
newspapers if you're more academically
focused you might think about an
academic journal television radio
there's really all sorts of ways to do
it and really the life want the lifespan
of a poll sort of goes in phases the
immediate results are of interest
especially to newspapers and news
outlets then they become something more
interesting to academics in those who
are sort of policy wonks after that
because the analysis gets more
complicated and it gets more in-depth
and it doesn't necessarily fit well with
the sort of immediate nature that comes
with publishing or it comes with news
the news cycle so after the initial
lifespan of the poll it sort of evolves
and it becomes interesting to people
like me and people like you students who
might actually work with the data we
have a poll how do we read it so here
we've got from about about a week ago or
so two weeks ago I guess is when I
pulled this pulling data from a number
of agents a number of polling agencies
and the results they're in this is
comparing Biden versus Trump in the
general election how do we read what
we're looking at here first thing first
this is a general election matchup and
it is a national matchup right this is
not some state this is how they would do
with this national sample of people so
first thing we want to do is look at the
source do we understand who the sources
is their name means something to me is
this reputable because not every survey
that's out there has been done in a
reputable way and if you're not who sure
who they are it would probably make
sense to do a little bit of research on
them in this case we're looking at NBC
News in the wall
Street Journal these are two
organizations have been in the polling
industry for quite some time I think we
can trust them in fact most of these I'm
very familiar with I'm less familiar
with Emerson Emerson or IB D
survey u-s-a I understand their
methodology very well they're generally
a trusted agency you may not have heard
of them but I have and then Quinnipiac
saan here as well and yes it's true
Fox News is a trusted pollster I know it
must be hard for some people who don't
believe that but it is in fact true
they're a legit polling that they follow
all the standards to a port they are
trying to do this as well as anyone else
they're doing the best they can do now
keep in mind I didn't say it was a Fox
news.com poll where you go online and
you fill out those polls that are on the
front page it's true those aren't valid
but Fox News is a pollster is a
respected polling organization so know
where it comes from because it matters
then look at the date range and then
keep in mind when I told you about a
snapshot this is when the survey data
were collected so this was the time
frame you have to think back to yourself
as anything major happened in this time
frame that might change how these
results are in the future chances are
that's true right they're only as fresh
as fresh can be and then they're no
longer really representative anymore so
for these dates were good then look at
what was the method and to demystify
what this is that 720 is the number of
respondents there is no sort of perfect
number of respondents I can tell you
about the fact the matter is it's gonna
be different based on what you're
actually trying to say what your
population is in this case 720 is fine
720 is a perfectly valid number the more
you have the smaller your margin error
is going to be but there is a line of
diminishing returns on this and it
happens somewhere between 1200 and 1500
respondents so you'll see that a lot of
news agencies opt for this number of
about somewhere between 5 to maybe 800
or so and that's that's a generally
pretty
most effective strategy of getting a
smaller margin of error and spending
money so this is 720 and then that RV
after it means registered voters alright
so this is not a likely voter sample
this is a registered voter sample there
are differences between the two just
because you're registered to vote
because does not mean that you are
actually going to vote but it depends on
what you're interested in at this point
of time and perhaps at the earlier parts
of the election cycle you do want to
know registered voters are thinking me
personally I like the likely voters but
it's a little bit more expensive and a
little more difficult to get the likely
voters because there are fewer of them
there are registered voters then you
look at the margin of error and this is
3.7 we generally look at this as a plus
or a minus effect and the best way to do
that is to sort of dig into what is
actually going on with this statistic so
those of you who thought you get away
without any math today are in for it
so statistics are probabilistic let me
say again statistics are probabilistic
but they're not deterministic I can tell
you something about a statistic and mean
that there's a chance that this is going
to happen and that chance is pretty good
do not bank on it the margin of error is
an estimate of that certainty how
certain I am I of my probabilistic
statement so we usually take this to
mean and most standard forms 95% certain
it's pretty good I would take 95% all
the time how do I interpret that 95 out
of 100 times I would expect the same
result that's what it means so that
means that 5 out of 100 it won't that
means that even with the best
statistical power I can give you if I'm
using the 95 percent threshold five out
of 100 times I'll be wrong that doesn't
mean the statistic was wrong it doesn't
mean my methods were bad it just means
it was one of those five it happens but
you know if I were to put it to you this
way
if I told you I could predict a roulette
table out how come 95 at a hundred times
you would take that bet every time even
if I was wrong some of the time even if
I was wrong five times in a row and I
asked you do you want to take my
roulette prediction your answer would be
absolutely now if I was wrong six times
zero you might start to question maybe
I'm just some sort of hustler who's lied
to you and taking some of your money
which is a reasonable reasonable thing
to say but in this case if I can
actually do this you would take it and
you'd walk away with and probably be
arrested by the casino so this is how it
works the margin of error is calculated
based on this formula and yes it's math
I know look at that square roots awful
um let's unpack this a little bit shall
we
P hat yeah it's got a little hat on
means that it's a sample proportion that
little hat means it's a sample not a
population proportion that's all it
means it's not that complicated it's
just something the statisticians do to
make sure they're clear about what
they're talking about so it's a sample
proportion okay and this is the number
in the sample we know that I can give
you that number all right it's not that
complicated I can give you the
proportion number because I just can't
tell you like if I go back to the the
Biden and and and the Trump statistics
there's a proportion there right that's
how much percentage did you get at the
two party vote very simple you can start
plugging these in as I tell my students
if you've ever baked a cake or have you
ever followed any sort of recipe at all
that's all this is this is a recipe but
it's in mathematical form you just start
baking the cake and you'll get the
answer Z then is that critical value for
the desired level of confidence now I
think you know it's always pretty
standard and I always use the 95%
confidence right that's good enough like
I said I take the I take the roulette
challenge every time and I'm just
telling you this but you could look this
up the multiplier when Z is 95 is 1.96
so now we have all the elements we need
to figure out the margin of error
all right so let's say we had an example
here
whatever proportion I was calculating
between two candidates one of them is
getting 50% of that I have a sample of
1,500 I have given you now everything
you need right 50 goes where the P hat
is once and twice at the top dividing by
the N which would be 1500 and then once
I'm done with that I multiply that by
the Z which is 1.96 and that gives you
the margin of error that's what you
would do if you were writing this out
mathematically with all those numbers in
there shortened down to that so 2.5 3 or
what we'd say plus or minus 2 point 5 3
and the plus or minus is important but
this is how it would play out in we'll
actually use the binding example for
this one so biden had 52 point one
percent of the vote according to this is
the national sample compared to 41 point
9 for Trump this is how you would think
about it if you were just plotting this
out in an XY axis very straight forward
you'd put you'd plot the 50 and then
your sorry sorry I was reading the
average there what I meant to say was
the biden has 50 Trump has 41 you plot
50 and then you would add according to
them their margin of error of 30 point
seven so the upper range of biden
support in this poll is fifty three
point seven right that would be go
mobile that there fifty three point
seven now if you want to figure out the
lower bound within 95 percent confidence
we're fine and support is you subtract
three point seven forty six point three
then all you gotta do just put Trump's
number there you plot at forty one do
the same thing plus three point seven
minus three point seven now you have
ninety five percent confidence bounds
for both and what we should be doing in
our head and if you want to you can
always draw this out the way I did it's
very easy
think about those lines those dashed
lines in between them where I'm putting
the top end for Trump and I'm putting
the lower end from Biden and I'm
shooting out little dotted lines into
space from there and I'm looking to see
if they cross over or if there's space
between them and there's space between
them on this poll which means that if
the poll were conducted ninety five out
of a hundred times we would expect Biden
to be in the lead that's what that means
please note I did not say that this
means Biden's going to win the election
it does not mean that it will be this
exact number it means that if we were to
do this poll ninety five more times but
the same methods with the same questions
with the same sampling frame I would
expect Biden to fall between somewhere
between fifty three point seven and
forty six point three 95 out of 100
times I would mean I would think that
Trump would fall somewhere between forty
four point seven and thirty seven point
three and those same instances in both
of those cases and both examples Trump
however would be losing in quotes to
Biden on this national level sample but
that's how you read that and that's what
that means
it does not mean that I figured out the
election and you should go and celebrate
or not to celebrate whatever you happen
to be hoping the outcome to be so
national polls are fun I enjoyed them
you enjoy them they're instructive it's
cool it's cool to talk about who's
winning horse races are in fact why most
of us like politics so much because it
really does come down to two people and
you know we we've partitioned ourselves
in in camps where we're Republicans or
Democrats and we route really hard for
our teams now and we've hardly for our
teams and we ever have before and that's
fun but it does very little to help us
understanding who's gonna win the
election right so that's where you have
to go to the state level
here's what it looks like about two
weeks ago between Trump and Biden Iowa
right
Trump is either depending on who your
which one you're looking at you look at
the time Siena or Emerson one or two
points ahead okay we don't need to draw
these out see this but we've got three
point one and three point two
respectively for their margin of errors
they're well within it which means that
statistically speaking 9,500 times we're
gonna basically be in a dead heat right
that's what we should expect to find
five out of those hundred we would
expect there to be enough separation
that would be outside of those maybe
maybe you get something outside the
margin or probably not so I always
competitive
that's one state do you have to do this
for all the other ones that matter some
of them needed skip I can skip New York
right I have a bold prediction for 2020
I'm calling it now the Democrats gonna
win New York all right
I know it's unpopular but that is my
prediction and I stand by it all right
but every other state that does have
sort of an impact on what the final
tally would be that are toss-ups at this
point Iowa's one of them I do want to
add though states are harder to do
there's fewer people in them especially
when you get down to a state the size of
Iowa it's a it's tough all right coming
up with a true predictive model of how
each state is gonna go I don't I would
not want to make my living doing that um
it's it's difficult to do because most
of them are within that margin there
right so yeah Trump is winning now could
he be winning in two more weeks could be
winning now I don't know maybe maybe not
it's too close so this is a toss-up this
gets into the big question right who's
gonna turn out to vote
that's what it always comes back to but
the only information I can take away
from this at least at this point in time
is I'm not going to put Iowa in Trump's
Trump's camp yet I'm gonna say it's
close and we'll have to watch carefully
come election night so if you yeah this
is good
you know but this is what it looks like
with weight when you when you jump into
the pole this is not uncommon so what
this is is the New York Times see an
ecologist is that Paul from Iowa
registered voters and it might be a
little hard to see but this is what it
looks like if you actually click on that
link and it takes you to a PDF and this
is where they're getting all their
numbers from they're breaking it down
and this was a really really nice cross
tabulation that they gave us I would say
a lot of other pollsters don't do this
they don't really give you a lot of
cross tabulations but they did here and
so we can actually do a deeper analysis
on this than we could have if it was
just the top lines which is what we were
looking at so we can look at things like
gender differences you know men versus
women and they definitely going to vote
for Trump category there's a 15-point
gap all right fewer women they're gonna
definitely vote for Trump or you go down
to definitely gonna vote for Democrat to
be named later there's another gap 22%
for men
40% for women that's an 18 point gap so
what would you say about this you would
say that if you were running a campaign
in Iowa that your message is going to be
better tailored if you're a Democrat to
the interests and the policy positions
that more women tend to hold because
that's where you're likely bases and
that's who you should be talking to
there's such a gap for men that you're
probably better off
I'm not saying ignoring them but
thinking about messaging in the right
way
race and ethnicity white and Latinos
again 13-point gap and the most likely
are definitely going to vote for Trump
gender broken down by here we're looking
at the horse race Joe Biden versus
Donald Trump for Donald Trump he gets
54% of men
37 percent of women 35 percent of men
for Joe Biden 53 percent of women
pation another area that's been
discussed at this particular matchup
lesson high school tends to vote for
well there's the you actually they split
out pretty well at high school or less
but as you get down to bachelor degrees
and then you get to postgraduate degrees
you start to see some significant
separation so Iowa is actually sort of a
difference it's different than a lot of
states here they don't follow the trends
of college degrees actually Trump does
pretty well so things to keep in mind
again a poll is just a snapshot in time
right it's only as good as the dates in
which it was collected the world changes
so fast now especially with the kind of
news cycle that we have I wouldn't trust
those numbers beyond a couple of weeks
samples are often imperfect that
introduces additional randomness Orinda
randomness can be considered noise in
which case the signal is harder to see
so more people saying I don't know or
they're making selections based on
imperfect information or perhaps they
really aren't sure but they're giving
you an answer anyway these are all
things that happen they happen
frequently and sometimes your sample is
to blame polls are probabilistic not
deterministic I cannot say this enough
we place too much emphasis on what a
poll can help us predict when in fact it
really is about what is the likelihood
of this occurring and it's never 100%
and then finally be on the lookout for
things like house biases and polls it
does happen but it's usually tied to
somebody's sampling frame the one that
sort of sticks out in current public
opinion polling is Rasmussen Reports
they tend to have a Republican bias and
I couldn't tell you exactly why except
that I know it's some some mechanism the
way they sample individuals but without
looking at their methodology fully I
couldn't tell you exactly what's at play
but they certainly do tend to pull a
little bit higher for things like
approval ratings and in the head-to-head
matchup
you can generally see this up pretty
straight yeah it pretty easily trends
trends are very useful for understanding
public opinion because they expand
beyond what is happening in a snapshot
in time in fact that's what a trend is
it's multiple snapshots typically done
with the same methodology over many
points in time so you can capture
changes and those changes can be
meaningful there's still noise because
they are a collection of singular polls
nothing's going to be perfect but we're
looking for a clearer picture and I
think trends do that so here's an
example I know it's probably impossible
to read but you can look at the galloped
public opinion website and play with
this is actually an interactive map but
just to tell you what you're looking at
here you're looking at the far left of
the top you're looking at Truman's
approval rating over the entirety of his
presidency right so clearly after FDR
passes away he comes into office he has
a very high approval rating and then
there are highs and lows of that
presidency Eisenhower may be perhaps the
most liked on average president over the
entirety of his period generally spends
most of his time in the positive range
Kennedy gets to spend all this time in
the positive range although he only had
three years right so chances are that
would have been different if time had
gone on but Johnson gets a bump and then
ends up actually bouncing around
not nearly as unpopular as you might
have guessed if you were thinking about
you know the way that we typically
portray the 60s Nixon well fairly
popular what happened to that guy and
then Ford you know comes in actually
with a reasonably good honeymoon
considering where he started but then
you know kind of bounces around the
middle Carter comes in higher down and
then ends very down so pretending the
the election of Reagan Reagan up and
down but Jen starts to really point to
the beginning of the long march to
polarization
that we are now living in today and that
Bush gets highs through the the first
Gulf War then hits a down economy and
these poll numbers decline Clinton up
and down and then the boom during the
90s of the the last part of nineties
goes up although you can kind of see the
effect at the impeachment trial and
there you see where it goes down that is
that is significantly after but it is
there and then Bush obviously George W
Bush gets incredibly high numbers after
9/11 but you can see the slow march of
time bring them down to some of the
lowest numbers we've seen Obama well
sort of what you would expect and this
is where you get to trump
right and this is what people are
talking about that Trump has never
enjoyed numbers above 50% he's the only
president that's done that and this is
what they're talking about trends right
that's not to say Trump hasn't had
better moments and worst moments right
you see the fluctuation on the far right
but it is below 50% and then if you
unpack that in the bottom there we can
really sort of see it over time it kind
of looks like a almost like a steady
heartbeat up and down up and down up and
down up down all the way through but
never about 50 it's it's actually from a
political science standpoint is sort of
remarkable trends are also useful for
discussing issues right so here's a gals
data on same-sex marriage all the way
through 2019 as you can see not not not
too long ago
yeah just 20 years ago right minority of
support here is 2015 right passage or
the legalization of same-sex marriage
it's only gone up since then all right
so the interesting thing about this is
you would ask is this just part of the
natural trend upwards or is there
something about the fact that same-sex
marriage is legalized people sort of
accept it become used to it and then
public opinion follows it's a chicken or
egg question I don't know
I think with marijuana legalization
though it's probably actually pretty
clear what's going on there's medical
California right 96 reefer madness does
not come to pass people realize they're
not gonna go crazy from smoking pot for
for medical reasons and then you get the
passage of recreational right so finally
it's that 50% on average tipping point
and look at it go all right look at it
go and so more and more we're getting
data out of the states like Colorado and
all the others have passed it since
Colorado has even point out to to my
students to remind them that
recreational marijuana is actually legal
in Alaska one of the most conservative
states in the Union to show how popular
of an idea it is now so challenges well
we talked about this a little bit with
cell phones cell phones are harder to
get they have an age bias to them and as
people like me yes me who's never had a
landline of their own move into the my
age group moves into more and more of
the majority of voters of the plurality
of voters I should say this makes public
opinion research even harder so the fact
that most of you don't even have a
landline anymore means that measuring
public opinion is going to continue to
face this problem and who answers their
cell phone from a number you don't even
understand nobody so you send it to
voicemail
it's a real issue and that is robo calls
right so probably how many guys are on
the Do Not Call list does it work no
I've gotten probably this week five
calls from the Social Security
Administration informing me that my
account has been suspended and I need to
call them right away otherwise well it
wasn't really clear just said to call
them it doesn't seem like a live person
so you know that happens and it's
happened a lot you know you used to be
the case you would get a
call and you get a call from somebody
like the Gallup Organization they'd say
we really want to talk to you about your
opinions and you'd actually be kind of
excited to weigh in nobody wants to talk
on the phone anymore and part of this is
because those companies that call you
now are telling you it's an opinion
survey but it's not it's actually them
trying to sell you something
are you currently happy with X Y & Z oh
you're not oh just so happens I happen
to work for this company would you like
to hear about the product I have to sell
you we've gotten wise is the problem
we've gotten wise to our phones and we
know that it's not usually a call that
we want to answer and so that coupled
with just the over saturation of this
everybody has a poll everybody's calling
you all the time and it's just something
people don't want to do anymore we're
also living online now this is where we
have our meaningful interactions so it
makes sense we wouldn't probably forever
call people on the phone and ask them
about things on the phone in a place
that's sort of weird right the phone is
for calling mom or mom calling you is
probably the case but you do everything
online you talk to friends online you
message online it makes sense that we
would eventually move to measuring
public opinion online and that's where I
think things are going but not everybody
lives online in the same way right there
are disparities about who has access who
can pay for access and then what people
are doing with it as well and as we're
finding out more and more the way in
which young people are engaging online
evolves and so public opinion will have
to evolve with that to you know forever
it was you go to a website well now
people don't even like going to web
sites anymore they'd rather have it in
an app they'd rather have it in text
message whatever it is it's changing
this constantly bringing up new new
challenges and the last one here I think
this one that falls under the radar a
little bit with most people
I did want to spend a moment talking
about it over the last couple years we
have gone through a pretty significant
political battle over how we're going to
run the united states census that battle
has been particularly over measuring
whether or not somebody is in the United
States legally I'm against it I'm
against the question and it's not for
political reasons it's from a research
standpoint the United States Census is
the best survey we have of everybody in
America it's how we know who we are is
how we know we're made of and ultimately
every assumption that we make about what
the public thinks is rooted in the
United States Census if we get it wrong
every public opinion poll will be wrong
after it so it's important that we get
it right and so I'm happy the question
didn't get on there from a totally
non-political perspective because it's
important for people like me and
everyone else who does work and if you
like knowing exactly what the public
thinks it's important for you to betting
markets probably should have pulled this
today and seeing how how maybe Duval
Patrick was doing in the betting markets
these are fun but they actually had some
truth to them because this is real money
right people are actually buying this
like they're buying stocks and you can
go I mean I think it's legal for
Americans I'm not gonna condone it so I
don't know you should do your research
if you're gonna do this certainly you
can do this if you're a British citizen
you can bet on the United States
presidency and so this is the couple
weeks ago betting market I'm one of the
there's multiple so you know betting
market for for the Democratic nomination
Elizabeth Warren is the most expensive
right and if you're gonna vote against
her or buy against her that 67 cents
Joe Biden only cost 23 cents so he's
probably a particularly low point
actually seems like a pretty good deal
looking at the data today
Buddha jesh 18 cents and you know
Sanders 14 Hillary Clinton 10 cents yeah
that's what I said she's not running but
maybe I don't know the betting markets
disagree betting markets are not
counting her out and if you want to get
in early 10 cents that's what it'll cost
Andrew Hayes got nine cents Club which
are four cents and Bloomberg was on
there two weeks ago three cents so you
know there was a there was at least a
couple of a couple of news headlines
that came out that suggested that there
was some rumblings out of the blue
Bloomberg can't so that's probably what
some people said but keep in mind if
your candidate doesn't get the
nomination you don't get anything
so that wisely I guess or dump it all
really
and that gets in so those are fine like
if you're looking for more source of
information things like those are out
there and they do sort of help paint a
different picture probably it helps to
complete the picture I was really happy
that uh 538 had not taken down their
final 2016 predictions because I I
thought it was sort of instructive to
talking about whether or not we can
trust public opinion research and so
what about 2016 people will say well it
doesn't that show that polling was just
off I'd say no first of all it did get
the popular vote generally right and
I've already I think made the case to
you that it's hard to do state-by-state
analyses and so I went through and
clicked most of these and most of the
time they're within the margin there
right so even the public opinion
research that we did have which is
probably not enough in states like
Michigan in Pennsylvania told us that it
was close then we should have probably
always seen it as close conventional
wisdom told us that it would it would
break toward Hillary Clinton and that's
the real problem
we relied a lot on conventional wisdom
which we typically
you typically use what you know to make
decisions why wouldn't you but the real
problem with 2016 was likely voters were
hard to predict right there was and this
is where you absolutely have to go to
Trump's campaign and say look they
figured out something that a lot of
people didn't know and a lot of
conventional wisdom didn't tell us that
there were voters out there that they
could get that didn't usually turn out
to vote but the wood for him and that's
what happened and they also probably
knew that there were enough voters that
typically would vote for the Democrat
that weren't probably going to turnout
and that's what happened turnout it's
always turnout in this case I think
Trump team had a little better idea what
that looked like do this 95 more times I
don't know I'm Clinton might win 95 of
those I don't know it's hard to say
state-level contests or hard so can I
trust public opinion polls going forward
I like to say they got 2018 pretty right
right they predicted a pretty big wave
and the wave came in all right
big dozens of C's changing in Congress
but it is always important to go back to
the states because you look at the
Senate contest and there's a slightly
different story there right not all it
didn't translate into every state all
the time but on the average it did do
they match other leading indicators for
example we'd like to use things like
who's retiring from Congress it's one of
the better indicators we have early on
as to which direction the winds are
blowing politically and in 2018 just as
it is so far in 2020 a lot more
Republicans are retiring and the polling
would match in a generic ballot a very
plus six seven eight or nine depending
on which poll you look at Democratic
year in 2020 right now a year ahead of
the election so you know talk to me in
like 11 months so thinking about 2020 as
I'm looking toward understanding polls
what can I
incorporate into my knowledgebase
turnouts gonna be higher this is
obviously you know this is a
presidential it's gonna be higher than
it was in the midterms or any midterm
but it's gonna be higher even for a
presidential year why a lot of states
were close once the states closed the
cats out of the bag
turnout goes up everybody feels it
campaigns adapt to it everybody starts
to behave like it matters and because
they think it matters they're more
likely to vote 80,000 votes decide this
election it's never been closer
really and people this is widely
understood and it's going to have an
effect on turnout 2018 was the highest
midterm turnout in 19 since 1914 I don't
see why it would be any different the
person I trust on this is Michael
McDonald the University of Florida and
he says probably 65 66 or maybe more
that's gonna be a record since 1908 if
that comes to pass what does that mean
it means it's gonna be harder to predict
likely voters because there's going to
be new voters there's gonna be
inconsistent voters and there's gonna be
young voters new voters are people who
just never voted for they're not
necessarily young inconsistent voters
are hard hope they don't have a voting
history they don't necessarily have a
strong part as an attachment so they
become difficult new voters are probably
angry that's usually what drives people
to poll when they're mad they tend to
vote more if they're if they're actually
upset about something and they tend to
just like young voters break the
Democratic way at least in this election
angry voters well whoever they're angry
at is who they vote against but young
voters are probably gonna break
Democrats they usually do and angry
voters are gonna be angry because
they're voting against Donald Trump so
what does this mean headwinds for
Republicans tailwind for Democrats today
this is the best I can tell you today
but it's gonna be hard you know remember
always think about the state-level
contests we can't say much about what's
going on nationally because it doesn't
translate as we saw when we looked at
Iowa versus the national numbers it's
not a plus aid in Iowa it's plus one or
two Trump it's the opposite direction
that's a 10 or 11 points we depending on
which poll you're looking at the 20/20
is gonna be hard I still think the polls
will be able to tell us something but
there is that chance hanging out there
that we don't quite know what the likely
voter models gonna look at so that's
gonna throw a few wrenches into the
problem here so I'm now happy to answer
any questions you might have two plugs
here we have the revealing of the first
calico school pull that happens
immediately after this talk at 12:45 and
then I'm encouraged and I would I would
say it's you should check out the
Hofstra votes podcast on iTunes or
Spotify or I guess you can find the
hofstra.edu/events is
so good I'm glad I explained it all all
right I'll see you guys later
I'm happy to make bad predictions by the
way and you can just throw them away
once I say them if you can get you to
the microphone and ask the questions
yeah please a critical issue is voter
suppression that's in the case of
Florida there's suddenly in the states
where they think new young voters or
inconsistent voters are they're
challenging could you discuss that so
there's a couple things going on in
Florida I assume you're referring to the
most recent amendment that they passed
which was supposed to restore voting
rights to felons as I still understand
it it's tied up in the courts what has
happened is they did pass through it's
actually a constitutional revision
Commission put a number of reforms out
there just to catch the audience up and
one of them was to make it so felons
would be allowed to get their Voting
Rights Act back once they had fulfilled
their their debt to society there was
not clear legislation as is often the
case when you look at ballot measures
there's not usually clear specific
elements of implementation in that so
that was left of the legislature the
legislature interpreted it in a very
strict way and part of that strict
interpretation was to make it so if
there were any sorts of cost monetary
costs associated with it that they would
have to pay those back before being
restored their felon before it felons
will be stored their voting rights it is
tied up in the courts and I think the
last decision was actually against the
legislature so this may sort of settle
itself out before then but there is a
question about how this because it was
it was in the tune of a couple million
or at least a million I think was and so
that some that this would break for the
Democrats in a major way but
there's a lot of cases like this so you
look at felon rights or post post you
know felony brights versus say it
depends on the state there's no real
sort of guidelines federally on how to
do that but other examples of this have
been you know voter ID laws and in case
of Florida they've also tinkered with
like early voting and stuff like that
the effects can matter it will matter
most likely it's not a huge effect right
maybe the maybe the Voting restoration
might be we don't know we've never seen
it so we have no baseline but in close
races a little bit it'll be it'll be
important so it's Florida and be close
probably yeah so yeah it matters but in
in a case of a blowout election it won't
really matter but in Florida I think it
does and ultimately you know these
things so far and the Supreme Court
level they sort of deferred to the
states on them and the current
composition the Supreme Court's suggests
they probably will continue to do so so
the battleground for those sorts of
policies is at the state level you know
you get you get what you elect right if
you lack the conservatives to your
legislature you tend to get conservative
policy and that's what they have in
Florida they have a Republican dominated
legislature nor Republican governor so
we shouldn't expect it to change anytime
soon groups have taken a different
course than they've gone through the
courts and they've had mixed success
there so it will be battled out through
probably most of next year I don't have
a clear picture how it's gonna go at
this point sir are there circumstances
where it's in the best interest of a
special interest group to lie to a
pollster I guess I can ask you to give
me a give me an example what do you what
do you think for example if a candidate
has limited resources with the effect of
the poll helped them or even if they
have a lot of resources so like a push
poll or something like that is that
we're kind of thinking like they get
some sort of
well like for example Biden is ahead in
many polls but he seems to be affected
by any fluctuation by a point of cell in
terms of the media and in some cases I
guess a candidate with limited resources
may be knocked out or given a big push
by the results of a poll yeah so you
know in terms of qualifying through the
debates right which is really kind of
what the candidates of the lower tiers
are after because to Biden right him
coming at a 26 or 27 ultimately doesn't
matter he's gonna be in the next debate
that's enough to qualify but for those
candidates who are bouncing around 2 or
3 percent they do have an incentive to
try to game the system but if you're if
you're a pollster doing your job in in a
way that that would be approved by the
regulatory or the this sort of
professional organization which is the
Association of American a public opinion
research a poor then you would not
engage in that sort of tactic right you
would just run your poll normally so
could you game it yeah it would it be
hard I would think so I mean you'd have
a really hard time doing that now if you
were trying to chase it so there's an
interesting thing that comes up though
is I think about this and that's a
question about how much does polling
influence what people think so could a
candidate go out there and maybe bomb a
state with information that's false
about how well the candidates doing and
it's just not based in reality and then
do people respond that going oh wow I
didn't know that that person was doing
so well yeah you could probably shift
public opinion a little bit because
viability matters a lot in the eyes of
the voter right so if I think a
candidate is viable that's the first
thing that goes through my mind do I
want to spend my one vote for this
person if they don't even have a chance
and that's why you look for those sorts
of fluctuations where you start to see
someone tick up and then you see the
sort of snowballing effect because more
and more
people look at that going hey they're
getting on the bandwagon I think I might
do that too
the last time that happened and actually
ended in the way that sort of just
snowballed all the way to the nomination
was was Obama right so it wasn't and
really until right before the Iowa
caucuses that he started to get traction
in the polls so if I was a desperate
candidate I might Commission a fake poll
that makes me look like I'm getting 10
points or something like that and try to
disseminate that information most news
agencies would would know better but you
know certainly things like push polling
is happened before so we have the
primaries in Iowa and New Hampshire and
obviously those results affect the
subsequent primaries and I was just
wondering how polling and I guess public
opinion in those particular states ends
up affecting the general oh well the
subsequent primaries in general election
so we do have sort of an odd mix of the
states as our first primary state sorry
I mean why do we look to Iowa for any
guidance except on corn prices which I
trust them on and but it is it's sort of
it's sort of a strange State to start
with but and and then their process is
very strange on top of that right they
don't run a traditional primary to do a
caucus so it's not you have to really
really really be excited about your
candidate or really feel strongly enough
about participation that you get up off
your couch go down to the caucus area
and caucus for however long it takes to
produce a winner and and it just that
that's not that those are sort of weird
people right I mean that's you know
that's to be to be quite frank like most
most normal average sort of the spectrum
aren't doing that so what does Iowa tell
us well it tells us probably something
about the activists part of the party
and that we can take away where the
energy is maybe a little bit and that's
probably what
translates into future contests is it
representative of America no is it
representative of any particular party
probably not but it is information it is
the first information that we get and so
it does in fact play into those
questions of is this candidacy viable is
this something that people are actually
considering or in the case of really
what I think Iowa can tell us is where
some of the excitement in this party
because that's usually who sort of ends
up winning or coming in sort of a
surprise finish in terms of New
Hampshire their their traditional
primary they you know they'll be able to
tell us something about where some
candidates are standing I don't think
anybody really thinks the first two are
important for like a front-runner to win
although you don't want to finish so
poorly that it looks like you know you
lost but in terms of like say a Biden or
something like that if he finishes third
in Iowa I think that's a pretty good
finish for him if he finishes second in
New Hampshire I think that's fine South
Carolina not a democratic state but it
does tell us something about one of the
core basis of the democratic party so
there is something to learn there Nevada
again a little bit different but still
some piece of information in a caucus
state Super Tuesday is really kind of
where we separate the wheat from the
chaff right so that's but in if you're a
not very well known not extremely
well-funded candidate and the best you
can do is sort of go around and get your
name out there
you have to do well in the states you
have to build momentum right and so
that's several weeks that gives you a
chance to do that yeah I don't know that
it tells us who's gonna winner is gonna
be but it gives us some sense of what's
going on in the field so it's not it's
not not not bad information this is not
predictive information
so this is kind of a little bit of
asking your opinion but we've seen a
real preponderance of discussion about
polls discussion about kind of where
candidates stand in them do you think
that the media focus on that takes away
from actual substantive discussions and
turns it into more of a horse race and
more of something that's about
electability as opposed to about ideas
well the model you know the media model
is to is to get more people to pay
attention and so the function is to is
different than what maybe is actually
important right and what's the
undercurrent of what's going on so it's
natural for news media to focus on horse
race because that's quite frankly the
most headline-grabbing thing that there
is right that's what we ultimately want
to know who's winning who's gonna win
right so I would not fault the media at
all for doing that because that's what
everybody else wants to hear the concept
of electability is an interesting one
what can you really know about it this
far out I don't know
there are plenty
you know the Trump wasn't electable
either right so I mean you can't really
say a whole lot about what the concept
of electability is because the the
matchup matters so much and the election
matters a lot right what happens during
the election that actually leads some
percentage of voters who really won't
have made up their mind to ultimately
make up their mind so whether or not
some of these electable sure you know
that matters on some sort of surface
level but individual decision making is
not going to be wrapped up into whether
I think that are elected or not
especially once you get down to two
candidates right it'll for a lot of
people it's gonna be personal you know
do I like this person do I not like this
person some people will be a single
issue that drives them this is person
support the issue that I care about yes
or no or is it going to be some sort of
protest vote is it going to be some kind
of box of issues that that matter to me
there's
in pick any individual voter and they
have an individual reason as to why they
came to that conclusion so again I think
for the media it makes sense because
that's what you're not gonna spend all
day talking about you know what was the
pulse Abe well if you if you go down
three levels it says this no that's
that's not really what's gonna sell them
newspapers or get more eyeballs on the
TV
it's about short pieces of information
that people don't understand so the
function is different but people who
watch elections carefully and really
want to understand know to go deeper
than to just look at the horse race
polls okay thank you you got high so
given the current social climate how
much of a role does the Bradley effect
play and how do that does your average
pollster account for it well I don't
know that we have a good accounting of
the Bradley effect I mean I think a lot
of people thought it would be there in
2008 and it kind of wasn't I mean so
you've won you'd have to nominate one of
the people of color to be able to really
tease that out I would say probably more
than anything else I see a gender bias
and the stuff that I've seen lately more
than I see a racial bias it's probably
still there I don't know that there's
any way you can truly account for it
generally speaking you want to take
people at their word when they're
answering these surveys I will say
though that there is a difference in
mode right so if you're asking somebody
on the phone or in person there's that
personal interaction there so if I call
you on the phone and I'm asking you who
you're gonna vote for you recognize that
I'm another person and I have my own
beliefs and even though I'm gonna not
judge you I'm gonna try not to judge you
you can't help but think that what is
this person thinking about me as I say
these answers right and so if you say to
yourself well I'm voting for Trump
but I don't know I want to reveal that
to just some random person calling me
all right that I think is somewhat
likely and you might see a depressive
effect in the type of mode whether it be
for Trump or whether it be for somebody
of color for a woman and that's where
actually I think one of the more
distinct advantages of online polling is
is that you're not interacting with a
person anymore it's just a computer that
sends your information somewhere and you
never really have to engage with the
idea that what is this person thinking
about me and you know that your data are
being anonymized and so you're never
gonna get tracked back to this no one at
your jobs gonna find out or anything
like that and I think people are more
honest you know all this all the
research in public opinion on this
particular difference of mode have found
that people are more honest about things
that they feel insecure about or they
think are socially undesirable so
whether it's certain kinds of behaviors
like smoking or whatever that society
has said this is not something you
should do or if it's that you're
embarrassed about how you think
politically the online format actually
tends to capture a little bit more of an
honest picture of how people behaved
than they do in the phone and so you
know I look at some of the polls right
now and I wonder to what extent are
people holding back information and I
think they are a little bit is there any
concrete way to account for that so how
do you online polls be a track IP
usually the more reliable ones but they
could still be gamed how do you account
for that as a serious bolster so that's
a good question
and you're right there are formats that
so I've used them all in some fashion or
not at this point there are services out
there like MTurk right so this is
Amazon's Mechanical Turk it is a service
in which you can pay people small
amounts of money typically to perform
some
or tasks I've used it and I will not use
the data from it and it's because I know
that the majority of the respondents I
get are from out of the country yeah
that's because I have their IP addresses
and I can see that they're from in lots
of times places from Eastern Europe
places where they could probably
understand English but you know maybe
the job markets not so great and that
this is actually a decent way to make
money um so I won't personally use their
data and I don't trust it if you were
doing an experiment maybe and there are
ways to try to increase the quality of
that sample but there's always going to
be people who can work around that
right so they can use IP they can use
VPNs and things to pretend like they're
from America so I won't use that and
then there are other samples that you
can get that that tend to check out but
they're just they're much more opt-in
samples and you run into questions about
who's opting in they tend to skew more
democratic they tend to skew more female
than than what you would want in a
regular sample and then there are the
more high-quality sources of which you
know to give a plug to it what we're
talking about in about ten minutes here
is are those like you gods and Ipsos and
they actually know the people who they
are right and they're confirmed American
citizens
for example YouGov has 202 million
Americans and they're panel eight
million people all over the world and
these are verified people they're
recruited they're screened and so the
quality and knowing that that person and
one I never get that information that's
totally anonymized for me but they know
who it is so long as they're seen as
reputable and I see them as such you can
trust it but yeah just going out there
and running a poll you're gonna run into
these sorts of problems and it's a very
wise you to be aware of them
