U.S. President-elect Donald Trump speaks at his election night rally in Manhattan, New York, on Nov. 9, 2016. Photo by Carlo Allegri/Reuters
Editor’s Note: How did the pollsters get it so wrong? It’s a question that’s been on many minds since Nov. 9, when Donald Trump was elected the 45th president of the United States. Leading up to Election Day, the grand majority of polls, pollsters and prediction markets had predicted a Clinton presidency.
Economics correspondent Paul Solman posed this question to University of Michigan economist Justin Wolfers, who also writes for the New York Times Upshot, which had predicted that Clinton had an 85 percent chance of winning. For more, tune in to tonight’s Making Sen$e report, which airs every Thursday on the PBS NewsHour. The following conversation has been edited and condensed for clarity and length.
— Kristen Doerer, Making Sen$e Editor
PAUL SOLMAN: So the outcome of the election has been pretty humbling for those of us who follow the prediction markets, no?
JUSTIN WOLFERS: I’d just say it was a little humbling. I don’t want to overstate that case though. Remember the Chicago Cubs were two games behind in the World Series, and betting markets said that there was only a 30 percent to win the World Series. As history now records it, they went on and won the World Series.
Well, betting markets said the same thing about Donald Trump. They said that there’s some chance he could win; he wasn’t the likely candidate by any means. But just as sports history has seen upsets happen, so has political history, and Donald Trump will go down as one of the unlikely candidates who won, perhaps the least likely. But, you know, the truth is when the markets tell you something is a one-in-seven chance to happen, it’s going to happen one in seven times.
SOLMAN: And that’s I think what was so hard for people like myself and you who followed the betting markets to really fully internalize, don’t you think?
WOLFERS: Yeah, so when the market says something has, say, an 83 percent chance of happening, it’s like a lot of people use an internal shorthand — “Well, that’s near enough that I think it’s pretty close to a sure thing.” So markets weren’t telling you it’s not a sure thing; there’s some risk this won’t happen.
You know, I was surprised the whole time actually that markets weren’t more confident that Clinton would win. They were worried that the polls might not get this right. That something else was going on out there. They pointed to that very real risk, and that very real risk turned out to be something that happened, so we were surprised, but we should have been no more surprised than we were when the Cubs won the World Series.
SOLMAN: Sam Wang of Princeton said there was a 99 percent chance that Clinton would win, and that was days before and right up to the night of the election.
WOLFERS: That’s Sam’s forecast, it’s Sam’s forecasting model; the betting markets told me it was maybe a one-in-six or one-in-seven chance Trump would win. We can’t say after the fact who was right. So the analogy my colleagues at the [New York Times] Upshot would use is this is like a 37-yard field goal. Well, what do we know after the fact? We know the field goal missed. What we don’t know was it a 37-yard field goal attempt, or was it, as Sam Wang would have it, a field goal attempt from right in front of the sticks? All we know at this point is the field goal missed.
SOLMAN: When I first began to think that Donald Trump had a real chance was when the New York Times Upshot made that field goal analogy, and my favorite kicker from my team, the New England Patriots, missed field goals from a shorter distance than the odds were of Donald Trump winning. And at that point, I thought, “Hmm, I’d better realize that this is a possibility.”
WOLFERS: Yeah, so I think one of the things that we learn here — and this is a lesson for both the media and for social scientists — is how difficult it is to communicate clearly probabilities. One of the nice things we’ve done in recent election cycles is we talk a lot more in terms of probabilities than we used to, but you need people to understand them. So what my colleagues at the Upshot were doing was try to find a way of making it concrete and understandable. And maybe they succeeded with you, Paul. A missed field goal reminded you that sometimes one in seven chances happen. But it also clearly failed with a large number of people who said that they didn’t see this coming, who thought there was no chance, [and who after the fact thought] we should get rid of polls, we should get rid of polling, we should ban the laws of math altogether.
That says that when we told people it’s an 80 or 90 percent chance, they always thought it was a 100 percent chance, and we’re still going to try and convey the degree of uncertainty in a much richer way.
WOLFERS: The big difference here is that he’s going to get to continue to kick field goals, and he’ll do it over many seasons and kick dozens and dozens of them, and Hillary Clinton, she only had one chance. That one missed field goal is the end of her career. Though he might miss one today, he’ll get three tomorrow, and by the end of the season, you can see that he’s going to get 80 percent of his kicks. With Hillary Clinton, she only had one chance. With politics more generally, election forecasters only get one chance every four years. So we’d like to know that if we say something is a 90 percent chance of happening it’s going to happen nine out of 10 times, but just figuring that out means we’re going to have to wait 10 election cycles, which is another 40 years. By the time we hit there, we got a whole new public to educate again.
SOLMAN: What’s a better way of trying to communicate to people what a 20 or 30 percent chance of winning really means? When people would ask me, I tried by saying, “Hey look, I’ve got a little mole here on my finger, if you tell me it’s only a 20 or 30 percent chance that it’s a melanoma, I don’t sleep at night.”
WOLFERS: So one of the things is to make sure you talk through all the scenarios. So you describe not only the possibility this could happen, but you tell a story about how it could happen. And if the story sounds plausible, then it enters the realm of imaginability. When people can imagine something, it becomes a little more concrete. I think it’s also important for us to talk more explicitly about doubts. The biggest story of the campaign really was that Clinton was dumping Trump. It was just as important that we talk about our doubts and our hesitations and the things we didn’t know as well. And my colleagues at the Upshot, bless them, came up with this idea of describing the election in terms of you know a 34-yard missed field goal attempt, there was another certain point Nate Silver described it as having the same odds of the Cubs winning the World Series when they were down 3-1. When the Cubs went on to win the World Series, that’s something that got people thinking. They understood unlikely things sometimes happen.
So it appears that the numbers, the simple math is not enough. We’ve got to somehow use the power of storytelling, of narrative, of analogy. I teach my students, and some of them mathematically gifted, some of them respond to storytelling, some to analogy, and we need to make sure we describe uncertainty using all those different languages.
The post How did the pollsters get Trump’s win so wrong? They didn’t, says economist Justin Wolfers appeared first on PBS NewsHour.