Photo: Lars Gesing/CU News Corps
The problem with polls: Opinion researchers preach restraint for horse race journalists
ELON, N.C. 鈥� They are Donald Trump鈥檚聽raison d鈥櫭猼re聽and cause for major Hillary Clinton headaches: Public opinion polls are the prime driver of much of the 2016 presidential campaign trail news. Media scholars semi-dismissively call such reporting 鈥渉orse-race coverage.鈥� Who鈥檚 ahead? Who鈥檚 behind? Who鈥檚 going to win?
But as cable news anchors hyperventilate in joyful anxiety every time they get to announce a new set of numbers, those who provide the data actually preach caution and restraint.
Donald Trump v. Bernie Sanders in 2016? Hold your horses, pollsters warn.
Andrew Smith is the director of the University of New Hampshire Survey Center, whose CNN/WMUR primary poll just this week found that聽聽over the once seemingly all-but-crowned Democratic frontrunner Hillary Clinton in the first-in-the-nation state. But聽, he pleaded not to let numbers carry you away.
Earlier that day, he had appeared on CNN.
鈥淭hey asked, 鈥榃hat do the new polls mean for Hillary鈥�?鈥� Smith said. 鈥淭hey mean she has got some problems. But they are in no way predictive of what is going to happen.鈥�
Although the 2016 cycle so far has seen mocking aplenty of common political knowledge, Smith warned not to jump on the rise-of-the-outsider bandwagon quite yet 鈥� for one simple reason: Voters have four more months to make up their minds. This far out, much of a frontrunner鈥檚 popularity is name recognition.
鈥淚t is summer 鈥� why would voters worry about politics?鈥� Smith asked then, four weeks ago.
We may now be past Labor Day, the informal kickoff for the presidential race, but most of what Smith said still holds true today.
鈥淲hen voters actually start to say, 鈥榃e are voting for somebody who could become the president of the United States,鈥� is it going to be Donald Trump?鈥� Smith cautioned. 鈥淚 don鈥檛 think so.鈥�
鈥淭he old-time journalists, they are the ones who are good. They have been through this stuff before. They call me and say, 鈥業 have to ask you what is going on with these new polls. My editor wants something a little bit better than: The polls are kind of meaningless.鈥欌��
The underlying problem: Political reporters don鈥檛 often do political science
Dartmouth political scientist and聽聽sides with Smith. Both men make the case that in general, reporters need to take more into account relevant political science literature.
If they don鈥檛, Nyhan said, 鈥淚t leads them to making claims that are just unsupportable with evidence or data. They try to turn politics into a dramatic storyline that is fun to follow.鈥�
Nyhan regularly contributes to聽, a data-driven politics blog at The New York Times.
鈥淥ne simple example: Why is someone up in the polls? What do reporters do? They follow the campaigns around, and they watch their speeches. The story that is often written: They must be up because they are somehow connecting with audiences. But there is no cause-and-effect relationship. When the economy is good, the crowds are probably going to like the candidate quite a bit.鈥�
History is speckled with candidacies who blew up in smoke after a brief stint in the limelight. Remember聽?
鈥淚 am the wisest man alive, for I know one thing, and that is that I know nothing.鈥�鈥� Socrates
Past cycles do not necessarily account for a growing nausea among voters regarding all things political establishment. But because the implications of this relatively new phenomenon are still uncertain, pollsters warn to proceed with extra caution rather than conclude that the voter anger is inevitably going to doom the likes of Jeb Bush or Hillary Clinton.
鈥淚 do polls because I track them,鈥� said聽聽and a nationally renowned leader in his field. 鈥淭hey are nothing but this little snapshot to see how people change over time, to see how they gain or lose.鈥�
But wait, there is even more to the poll puzzle
But there isn鈥檛 just the meta-uncertainty of what voter disgust and the historic size of campaign pockets and the GOP field will eventually mean. You also have to consider many a technical issue with polling that can easily get lost in the rumble of the horse race.
鈥淚f you look at polling for the Iowa caucuses on paper, you would think it can鈥檛 be done,鈥� said the聽鈥檚 Ann Selzer, another of the guild鈥檚 most well-respected practitioners. 鈥淚f I weren鈥檛 already doing it, I would say, 鈥楴o, thanks.鈥� I said, 鈥楴o, thanks鈥� to Nevada for the same reasons.鈥�
Only a very small number of voters go through the multi-hour, mid-winter ordeal of caucusing for their candidate. Many general election voters 鈥� and their opinions 鈥� are excluded right out of the gate. Once pollsters weeded out those people 鈥� which assumes that they don鈥檛 change their minds and show up on caucus night four months from now nonetheless 鈥� the next set of problems already lurks just around the corner.
翱痴贰搁痴滨贰奥:听
鈥淵ou can change your registration on caucus night,鈥� Selzer said. In practice, that means not every Democrat and every Republican on the voter registration list (the basis for the Iowa Poll-sters) will actually caucus as such. Then there鈥檚 the one-third of the electorate who are registered as independents as well as those 17-year-olds who are not registered voters yet but who may do so on caucus night because they will be 18 come General Election Day.
鈥淚t鈥檚 also an over-polled population,鈥� Selzer mentioned yet another factor that can easily skew the results.
Sub-par poll question: 鈥楢re you from the South?鈥�
And then there are the actual questions that pollsters ask 鈥� and more importantly,聽丑辞飞听they ask them.
Kenneth Fernandez,聽聽in the presidential battleground state North Carolina, gives a specific example: determining what constitutes a 鈥淪outhern鈥� voter and its values.
鈥淭he South is affecting the rest of the country because Southerners have moved to the Midwest, and you have Northeasterners moving into the South to get away from the snow,鈥� Fernandez explained. 鈥淲e actually found that a better question is not whether you are from the South but whether you have a Southern accent. Our students will code at the end of the survey: Does the person have a strong, slight or no Southern accent?鈥�
But the abundance of polls also has an upside.
鈥淚n North Carolina, we were the only game in town for a long time,鈥� Fernandez said. 鈥淣ow you have lots of different organizations from inside and outside the state who poll here because it has been seen as a battleground.鈥�
He uses the example of Nate Silver鈥檚聽聽at The New York Times, which used public opinion polling to predict election results in all 50 states.
鈥淚f you have only one poll in one state, your data is just not going to be as reliable,鈥� Fernandez said. 鈥淣o matter how perfect a polling organization does things, you could be well outside the margin of error just because of random and sampling errors.鈥�
The bottom line: There is a whole host of issues that is beyond the control of polling science. And in the roller-coaster ride that the 2016 presidential election cycle has been so far, that notion might be truer than ever before.
***
Make sure to also follow聽.