When pollsters failed to see the surge of support for Donald Trump in 2016 and 2020, it seemed like a low point for the industry. But in fact, surveys are getting more accurate, as a look through the archives shows.
Category
😹
FunTranscript
00:00That says it all. It's the Trump.
00:02Trust in polls and pollsters hit a new low in 2016,
00:07when most showed that Hillary Clinton was more likely to win.
00:11How did the polls lead us astray?
00:13It just came out of nowhere.
00:15And then again in 2020.
00:17The polling was wrong. It was wrong again.
00:20Fast forward to 2024, and few are daring to make a call.
00:25I mean, this is the closest race in American history.
00:27It is really neck and neck.
00:29In fact, there is only one perfect poll,
00:33and that's called an election.
00:35Believe it or not, the polls are not becoming
00:37less accurate over time.
00:38So how did a once-respected industry
00:41that grew to a value of $20 billion come to this?
00:45And can it win that trust back?
00:47Fame statistician Dr.
00:49George Gallup tells Washington from week to week
00:51what the nation is thinking.
00:54It's been more than 80 years
00:55since a man named George Gallup conducted his first poll.
00:59He used a simple but effective technique
01:01to survey the opinions of the American public.
01:045,000 men and women mathematically chosen
01:06from every walk of American life
01:08to make up an accurate microcosm of the entire U.S. population.
01:12Before this time, surveys had been a lot less scientific.
01:17How are you going to vote?
01:18I'm going to vote for Roosevelt
01:19because I think he's going to bring back the ban.
01:22The most reputable forecasting of the early 20th century
01:25was conducted by the Literary Digest.
01:28It asked readers to mail in sample ballots.
01:31In 1936, though, the Literary Digest backed
01:34Alf Landon to win the presidential election.
01:37But Roosevelt ended up winning by a landslide.
01:41The great American public has its say at the polls,
01:44and the result is a Roosevelt victory of amazing proportion.
01:48George Gallup had predicted that result
01:51by surveying a wider cross-section of Americans.
01:54Today, the true cross-section survey
01:56has become a trusted political barometer.
01:59The Literary Digest went bankrupt,
02:01and Gallup's polls became a hit.
02:03The Gallup surveys of the 1930s
02:06ushered in a new era for polling and for voter confidence in it.
02:10Back then, Americans were not only willing
02:12but excited to talk to pollsters.
02:15It used to be such an honor to be included
02:19as one of the special households
02:21who got to speak for thousands of Americans.
02:24In 1948, Time magazine even described Gallup
02:27as the Babe Ruth of the polling profession.
02:31That same year, Gallup backed the wrong candidate
02:33in the presidential race, but his reputation survived.
02:37We know that no better method has yet been found
02:40for measuring political trend.
02:43Those were the days when polling was revered,
02:48and technology had to catch up with polling,
02:50which it finally did in the 1970s.
02:53Once telephones were widespread in people's homes,
02:55pollsters didn't have to go door-to-door anymore.
02:58But there was a downside.
03:00Telephone response rates began to fall in the 90s
03:02with the invention of caller ID.
03:05Some companies eventually transitioned to online surveys
03:08that voters can respond to whenever they want.
03:11And by the early 2000s,
03:13there were more than 5,000 polling organizations in the U.S.
03:17That's when a baseball statistician named Nate Silver
03:20analyzed a number of polls and fed them into models
03:23that simulated the elections.
03:25He used them to come up with forecasts
03:27and give the public a glimpse
03:28of what Election Day in 2008 could look like.
03:31And may God bless the United States of America.
03:34Silver was able to correctly predict the winner
03:37in every state except for one.
03:40Basically, he made polls cool again.
03:43But in the 2016 elections, all of this changed.
03:47Look at how close we are to the polls.
03:49Look at how close he is.
03:51Right now, he has 257 electoral votes.
03:54He needs 13 more.
03:56It wasn't just a tough night for the news industry
03:58but for pollsters.
04:00I had chills when it became clear,
04:04whatever it was, 9, 10 o'clock Eastern time,
04:06that this was no layup for Hillary Clinton.
04:09It was a huge shock.
04:11Dr. Wong, you tweeted recently
04:13that you were so sure of the result
04:15that you'd put it up on the screen, eat a bug.
04:18A lot of people were wrong,
04:19but nobody else made the promise I did.
04:21The polls basically got trashed.
04:24But that wasn't totally fair.
04:26Polls are not meant to be a exactly perfect prediction
04:29of what the eventual vote breakdown is going to be.
04:32And it's normal for the polls to be three to four points off.
04:34The polls did miss, but not by an unusual amount.
04:38They were off by about 4.8 percentage points.
04:42That's within half a point of the average error
04:44for election polls over the last 50 years.
04:47On election day, pollsters said that Clinton was leading
04:50by about 3.2 percentage points.
04:53Clinton did end up winning the popular vote,
04:55but by a smaller margin of 2.1 points.
04:59And as we all know, she lost the election.
05:01What happened was that Trump was able
05:03to win the electoral votes in a lot of states
05:05where Clinton was leading by a small margin.
05:08But fine margins can make a huge difference
05:10in the electoral college.
05:12Part of the problem was that pollsters were too focused
05:14on talking to those who seemed like likely voters
05:17based on their past voting history.
05:19The people who ended up voting for him
05:21are perhaps low propensity, low turnout voters
05:24who would not pass a likely voter screen
05:26or just not be reached by a poll at all,
05:28but did ultimately turn out to vote for him.
05:30They also didn't factor in education levels.
05:33College-educated voters were more likely to vote for Clinton
05:36and they were also more likely to respond to surveys.
05:40So by not adjusting the sample, pollsters ended up
05:42with a forecast that overestimated support for Clinton.
05:46You can end up in a situation
05:47where you just miss a big part of the electorate
05:49and end up underestimating a campaign support.
05:52Ultimately, the results didn't match the scenario
05:54the media had hyped up the most.
05:57But the reputational damage seemed to be done
06:00and the public started to lose faith.
06:02Get it done! Get it done!
06:05Pollsters tried to implement a couple of changes
06:07in the 2018 midterm elections,
06:10like factoring in educated voters,
06:12and the forecasts turned out to be correct.
06:15But when 2020 came around,
06:17lack of trust was still one of Jay's biggest challenges.
06:20Voters were reluctant to answer his questions.
06:23How do we draw a sample that actually is random?
06:28And we as a profession are struggling right now
06:32with how to do that
06:33because people do not want to be bothered.
06:36They have ways of communicating that are invisible to us.
06:40Trump voters are less willing to participate in the polls,
06:43but not because they're shy.
06:45If you have been conditioned for four years
06:49by tweet after tweet that says,
06:51that poll that shows me, Donald Trump,
06:55trailing is a fake poll,
06:57you may be reluctant to talk to a pollster.
07:01And any poll that the fake news gives you,
07:03because the real numbers are higher,
07:05but any poll they give you is a fake.
07:07Pollsters did correctly predict
07:09that Joe Biden would win the 2020 election,
07:12but they again underestimated support for Trump.
07:15One of the states that first started reporting its votes
07:17was Florida, where polls had Biden up by a few points,
07:20but it was still expected to be a very close race.
07:22As results started coming in,
07:24we saw that Trump was overperforming the polls again.
07:27News organizations chose to represent
07:29their predictions in new ways.
07:31The New York Times published a table
07:33that included a column that read,
07:35if polls are as wrong as they were in 2016.
07:39And even FiveThirtyEight showed a number of maps
07:42to tease the idea of multiple scenarios.
07:45Overall, the polls did all right.
07:47Between response rates,
07:48coming up with a representative sample,
07:50and accounting for the pandemic,
07:52given that, it is pretty impressive
07:54that polls did predict the right winner.
07:56They were off by four points,
07:57and that's normal by historical standards.
08:01More than 100 years after the first poll was conducted,
08:04the industry has had to reinvent itself all over again.
08:09Would you mind telling us
08:09who you're going to vote for and why?
08:11I'm going to vote for Governor Roosevelt
08:13because I think he'll bring back prosperity.
08:16I think Hoover deserves another chance.
08:18I'm sure he's the best man for that office.