Journalists, politicians and voters all rely on polls — and often, they’re seen as objective measures, impervious to the partisan divide. But what steps go into gathering the data, and what factors can affect the results?
These are the questions that four panelists attempted to answer on Sept. 25 in “Polling: art, science or guesswork?” Jointly hosted by the USC Center for the Political Future and USC Annenberg Center on Communication Leadership and Policy, the event explored the methodology behind polls, the problems pollsters encounter in interpreting them and the effect they have on elections, from the race for the presidency to the upcoming midterms.
“It’s important if you are a citizen of the world today, even if you don’t care that much about polling, to be able to just have a sense of what you can trust, who does good work, and what does it even mean?” panelist Jill Darling, Survey Director at the USC Dornsife Center for Economic and Social Research, said at the event.
She was joined in the discussion by Jessica Yellin, a former White House correspondent for CNN; Bill Burton, managing director for consulting firm SKD Knickerbocker; and Justin Wallin, CEO of J. Wallin Opinion Research. Bob Shrum, Director of the Center for the Political Future and a veteran political consultant, moderated the conversation.
President Donald Trump was widely criticized last year for alleging that polls showing he had a low approval rating were “fake news” and should not be trusted. But in reality, everything from how a polling sample is chosen to the way that questions are phrased can affect poll results, according to Wallin.
“You have to peel the onion a little bit and find out who exactly was researched, and if it was a representative sample,” Wallin said.
Darling said polls are most accurate when they’re conducted by a real person over the phone, rather than electronically or by voice message — and Yellin added that the networks she worked with, CNN and ABC, would not show the results of “robo-polls” on air. But Burton said that online polling is growing more accurate, and can actually help widen the poll sample if utilized effectively.
Polling played a crucial role in the 2016 election, when most major news organizations predicted that Hillary Clinton would win by a large margin. Burton said that a lot has changed since then, particularly when it comes to questioning the methodology behind polls rather than just accepting them as fact.
“The first thing you have to challenge is conventional wisdom,” Burton said. “Work against that confirmation bias … so that we’re not missing some of the warning signs.”
This was difficult for journalists during the election, Yellin said, because the sense of support for Trump that they noted on the ground were not reflected in the polls they were taught to trust.
“They were caught in a difficult position,” Yellin said. “Do I lean into what I’m seeing and experiencing, or into what the data tells me?”
For Darling, the problems with polls during the election stemmed from the fact that most polls asked different groups of people about their support for each of the candidates at various points in time. USC’s poll, which it conducted in conjunction with the Los Angeles Times, traced the opinions of the same group of people over time — seeing how their views on Trump and Clinton changed as events played out in the months leading up to November. It ended up correctly predicting that Trump would win.
In the 2018 midterm elections, USC plans to continue using this strategy — and it’s registering a swell of support for Democratic candidates among women, Darling added. For Yellin, one of the crucial deciding factors will be the effect that the Senate decision on Brett Kavanaugh’s Supreme Court confirmation will have on female voters.
“If there’s a sense that we’re having a repeat of the Anita Hill hearings, it could really galvanize women voters who otherwise might have stayed home,” Yellin said.