The Rise of the Poll Quants (or, Why Sam Wang Might Eat a Bug)

Tom Bartlett, 11/05/12

If you watched Meet the Press this past weekend, you learned that the presidential election was “statistically tied” and could be a “photo finish.” The Associated Press predicted a “nail biter.” The Philadelphia Inquirer threw up its hands, saying the vote was just “too close to call.”

Sam Wang begs to differ. By day, Wang is a neuroscientist at Princeton University, where his lab uses lasers to monitor the chemical signals of cells in the cerebellum. He co-wrote a recent paper that found that mathematics and science majors are more likely than humanities students to have a sibling on the autism spectrum.

But in the evenings, after his wife and 5-year-old daughter are in bed, the 45-year-old turns his attention to polls. Since 2004, Wang has used his considerable data-crunching chops to forecast elections, publishing his results on Princeton Election Consortium, a very popular blog with an extremely dull name. He expects to log around one million hits on Election Day alone.

His posts are engaging and often droll, but it’s the numbers that are the real draw. Wang, as I write this, believes that President Obama has a 99.8-percent chance of winning the election. No need for nail biting.

Actually, 99.8 percent is his Bayesian prediction, which refers to the branch of logic that uses statistics to determine probabilities. In another model, which assumes that opinions could fluctuate in either direction--what he calls random drift--Obama has a 98.2-percent chance of winning. The upshot: According to Wang, it is “highly implausible” that Mitt Romney will add the White House to his list of homes.

 In 2008, Nate Silver made forecasting elections nerd-cool. Silver was already known as a baseball statistician when he turned his mathematical talents to national politics. As he writes in his new book The Signal and the Noise: Why So Many Predictions Fail--but Some Don’t, he figured that someone “could look like a genius simply by doing some fairly basic research into what really has predictive power in a political campaign.”

He became that someone, starting a blog he titled FiveThirtyEight, after the total number of Electoral College votes, which has since been snatched up by The New York Times. His posts attract hundreds of comments, and he has more than 300,000 followers on Twitter.

Wang doesn’t have nearly as large an audience, but it’s been growing. He got into the game for pretty much the same reasons as Silver: He thought he could bring some statistical rigor to a discourse dominated by talking heads nattering on about the number of yard signs in a random neighborhood or the volume of cheering at a swing-state rally. “I’m use to solving the problem of taking a data set and extracting something when there’s a lot of noise,” Wang says. “That’s something I do every day in my work. The presidential race was something that was crying out for that.”

He learned from Silver’s quick success that audiences wanted more than statistical breakdowns. They wanted explanation, banter, personality. His posts are still driven by numbers, but you also get a dose of Wang’s sense of humor, like the way he has relentlessly mocked the notion of Romney’s momentum (“Ro-mentum!”) in recent weeks. He slapped John Dickerson, who writes about politics for Slate, for arguing that Romney was “peaking at just the right moment.” Wrote Wang: “Ah, yes. The Great Election of October 13, 2012. I remember it well.”

Like Silver, his posts attract hundreds of comments, but he moderates them to keep the discussion on track. Wang is not interested in fostering the kind of low-brow, partisan blather that’s the hallmark of sites like The Huffington Post and The Daily Caller. He favors quantitative comments, mixed with the occasional crack about the possibility that Wolf Blitzer is an android.

The newest kid on the forecasting block is Drew Linzer, an assistant professor of political science at Emory University, who started a site called Votamatic this past summer. He designed it as a companion to his published research and as a way of informing an audience slightly broader than the people who thumb through political-science journals in their spare time. He’s succeeded: Votamatic gets tens of thousands of hits a day.

So what exactly do these guys do? Basically, they take polls, aggregate the results, and make predictions. They each do it somewhat differently. Silver factors in state polls and national polls, along with other indicators, like monthly job numbers. Wang focuses on state polls exclusively. Linzer’s model looks at historical factors several months before the election but, as voting draws nearer, weights polls more heavily.

At the heart of all their models, though, are the state polls. That makes sense because, thanks to the Electoral College system, it’s the state outcomes that matter. It’s possible to win the national vote and still end up as the head of a cable-television channel rather than the leader of the free world. But also, as Wang explains, it’s easier for pollsters to find representative samples in a particular state. Figuring out which way Arizona or even Florida might go isn’t as tough as sizing up a country as big and diverse as the United States.”The race is so close that, at a national level, it’s easy to make a small error and be a little off,” Wang says. “So it’s easier to call states. They give us a sharper, more accurate picture.”

But the forecasters don’t just look at one state poll. While most news organizations trot out the latest, freshest poll and discuss it in isolation, these guys plug it into their models. One poll might be an outlier; a whole bunch of polls are likely to get closer to the truth. Or so the idea goes. Wang uses all the state polls, but gives more weight to those that survey likely voters, as opposed to those who are just registered to vote. Silver has his own special sauce that he doesn’t entirely divulge.

Both Wang and Linzer find it annoying that individual polls are hyped to make it seem as if the race is closer than it is, or to create the illusion that Romney and Obama are trading the lead from day to day. They’re not. According to the state polls, when taken together, the race has been fairly stable for weeks, and Obama has remained well ahead and, going into Election Day, is a strong favorite. “The best information comes from combining all the polls together,” says Linzer, who projects that Obama will get 326 electoral votes, well over the 270 required to win. “I want to give readers the right information, even if it’s more boring.”

While it may not seem likely, poll aggregation is a threat to the supremacy of the punditocracy. In the past week, you could sense that some high-profile media types were being made slightly uncomfortable by the bespectacled quants, with their confusing mathematical models and zippy computer programs. The New York Times columnist David Brooks said pollsters who offered projections were citizens of “sillyland.”

Maybe, but the recent track record in sillyland is awfully solid. In the 2008 presidential election, Silver correctly predicted 49 of 50 states. Wang was off by only one electoral vote. Meanwhile, as Silver writes in his book, numerous pundits confidently predicted a John McCain victory based on little more than intestinal twinges.

Simon Jackman thinks we’re in the midst of a shift in the world of political predictions. His poll-averaging model, Pollster, which is published by The Huffington Post, is forecasting a narrow Obama win (277 electoral votes). Jackman, who is a professor of political science at Stanford University and author of Bayesian Analysis for the Social Sciences, thinks that models like his and the others present “a real challenge to the conventional great oracle pundit.” Most journalists are ill equipped to interpret data, he says (and few journalists would disagree), so they view statistics with skepticism and occasionally, in the case of Brooks, disdain. “The data-driven people are going to win in the long run,” Jackman says.

He sees it as part of the rise of what’s being called Big Data--that is, using actual information to make decisions. As Jackman points out, Big Data is already changing sports and business, and it may be that pundits are the equivalents of the baseball scouts in Michael Lewis’s book Moneyball, caring more about the gracefulness of a batter’s swing than whether he gets on base. “Why,” Jackman wonders, “should political commentary be exempt from this movement?”

That’s John Sides’s point, too. Sides, an associate professor of political science at George Washington University who blogs for The Monkey Cage and has written for FiveThirtyEight, is exasperated by articles that speculate on momentum and refer vaguely to polls. “Some polls say? Which polls say?,” Sides exclaims. “It’s not that all of politics can be explained by science. It’s just that, for God’s sake, we can bring a little bit of science into our analysis of politics.”

Last week the professional pundit and MSNBC host Joe Scarborough ranted that people like Silver, Wang, Linzer, and Jackman--who think the presidential race is “anything but a tossup”--should be kept away from their computers “because they’re jokes.” Silver responded by challenging Scarborough to bet $1,000 on Romney (in the form of a donation to the American Red Cross) if he was so sure. This led to hand-wringing about whether it was appropriate for someone affiliated with The New York Times to make crass public wagers.

But the bet seemed like an important symbolic moment. The poll aggregators have skin in the game. They’ve made statistical forecasts and published them, not just gut-feeling guesses on Sunday-morning talk shows. And, in Silver’s case, as a former professional poker player, he is willing to back it up with something tangible. Alex Tabarrok, an economist and blogger for Marginal Revolution, applauded, calling such bets a “tax on bullshit.”

Wang is willing to put it on the line as well, albeit in a more gustatory manner. If Romney wins Pennsylvania, a state many pundits have called a tossup, he wrote on his blog that he will “eat a bug.” If Romney wins Ohio, he will eat “a really big bug.”

He promised to post the photos, too.

What will the poll quants be doing on election night? Linzer says he will be “watching TV with my laptop and ignoring my friends.” Wang doesn’t really enjoy election night because the information dribbles in, and nothing is entirely clear until the next morning. It’s frustrating for a numbers guy. If he had the willpower, he says, he would avoid the spectacle and go to bed early. Instead, he will probably do what he did in 2008. “I was with a bunch of people who were Obama supporters. They were very pleased about Obama being ahead on election night, and I was concerned that early Virginia returns didn’t match my projections,” he says. “That tells you something about how quasi-autistic we are over here.”

http://chronicle.com/blogs/percolator/the-rise-of-the-poll-quants-or-the-demise-of-the-tv-pundits/31636