How Top Pollsters Grade 2024’s Polls

Just after Election Day, Charles Franklin, the director of the Marquette Law School Poll in Wisconsin, received a lot of requests for media interviews.

“I got at least three, if not four, reporter interviews premised on: ‘The polls were wrong again. How bad were they this year?’” Dr. Franklin said.

But overall, the polls had a pretty good year. In five of the seven swing states, and nationally, the final averages of polls aggregated by The New York Times missed the actual presidential election margin by less than three percentage points. All of the swing state results were within the typical margin of error.

Yet Donald J. Trump’s sweep of the swing states and decisive Electoral College win, along with his popular vote victory, left many voters with the feeling that the polls missed the mark by not conclusively pointing to a Trump win.

The Times talked with six top pollsters, who graded the industry’s performance this year between a C+ and an A- (their analyses were roughly the same; some were just admittedly tougher graders).

They are largely optimistic about the future. Each election cycle provides a new opportunity to learn and tweak polling methods, they said. And political polling has always attracted ire: In any given poll, somebody is bound to be unhappy to see the results. Here are some of their biggest takeaways from 2024 and where they see the industry heading.

Polls did pretty well in a challenging year.

Anyone in the polling world knew heading into Election Day that a Trump victory was entirely plausible. The polls, on average, showed a very tight race between Mr. Trump and Vice President Kamala Harris. Considering the margin of error, that meant either candidate could be poised to win, and even sweep the swing states.

Because the election results were so close (Ms. Harris was within 2.2 percentage points of Mr. Trump in four of the seven swing states), it was difficult or impossible for those interpreting the polls to “call” a winner.

“Polls are a helpful indicator of where the election is headed,” Dr. Franklin said. “But they’re not an infallible crystal ball.”

That can make polling in a close election — as the United States has had in the last four presidential races, with popular vote margins within five points — unsatisfying. And polls that missed by a wider margin but showed Mr. Trump in the lead might be interpreted as more accurate than those that missed by a smaller margin but showed Mr. Trump and Ms. Harris tied, or Ms. Harris slightly ahead.

“If we have a 50-49 race, and the outcome is 50-49 either way, we say ‘got it,’ even if we’re on the wrong side,” said Douglas Rivers, the chief scientist at YouGov. “If people expect polls to be accurate to a point, they’re deluding themselves.” Polls can be accurate to within about three percentage points, he said; anything more precise than that is outside the scope of current polling capabilities.

But polls in a close election can still offer insight. They can help explain the narrative of the election ahead of time (in this year’s case, that it was a close race); they can capture important shifts in voter behavior; and they can give greater insight into the results after the fact. This cycle, for example, voters consistently said the economy was their most important issue, and they consistently said they trusted Mr. Trump more on it — a simple finding that helps explain why he was triumphant.

“I think our bigger problems as an industry have more to do with how the community is being perceived in an era where there is significant doubt about what it really does and polling’s role in elections,” said Lee Miringoff, the director of the Marist Institute for Public Opinion.

But the polls underestimated Trump, again.

As close as the race was, Mr. Trump won. Across blue states, red states and swing states, the polls systematically underestimated Mr. Trump’s support. Polls are always slightly off in one direction or another, but that they’ve now missed in the same direction three cycles in a row is an indication that they are struggling to capture Mr. Trump’s voters in particular.

On average, the polls underestimated Mr. Trump’s support by about two percentage points, a smaller error than in 2020 and 2016. Pollsters attributed this slight improvement to some of the changes made after those past misses, including weighting (adjusting the share of responses) based on education and past vote, but said they clearly weren’t enough.

Support for Mr. Trump has been hard to measure, in part because he does well among less engaged voters who don’t participate in polls. “We have pretty definitive proof now that we have a non-ignorable nonresponse problem,” said Clifford Young, the president of Ipsos Public Affairs. “We have these individuals who typically don’t participate in anything, and in years past, decades past, it didn’t matter if we didn’t poll them because they wouldn’t show up anyway” to vote, he said. (He gave the industry a C+.)

What’s not clear is whether this nonresponse issue extends to races where Mr. Trump isn’t on the ticket. In the 2022 midterm races, for example, polls had their most accurate cycle in decades. Going forward, the shifts in methodology may prove sufficient for capturing Republican voters in a post-Trump era.

“We still obviously have challenges,” said Ashley Koning, the director of the Eagleton Center for Public Interest Polling at Rutgers University. “The issues we’ve been seeing, are they because of Trump, or is it something bigger? Have we entered some kind of new era of polling?”

Polls will continue to evolve, and that’s OK.

The country’s leading pollsters once all used very similar techniques, randomly dialing phone numbers to find respondents. These days, with landlines disappearing and fewer Americans willing to answer calls from unknown numbers, pollsters have started to diversify their approaches.

Those that still make phone calls mostly call cellphones now, and many include online panels. Some still use live interviewers, others use robocalls to conduct polls, and many pollsters include some combination of all of these techniques. There was even some experimentation this year with using artificial intelligence to design survey questions and conduct interviews.

“I don’t think it’s a bad thing that we have different methodologies — it kind of makes the aggregate numbers more realistic and more reliable,” said David Paleologos, the director of the Suffolk University Political Research Center. “What happens is you’ve got different researchers coming at the same research problem in different ways.”

Ultimately, a full picture of which approaches were most accurate this time will not come into view until researchers can access updated voter files (typically in January) and compare actual voter behavior with pre-election polls.

As the industry evolves, Dr. Rivers said, top pollsters’ “north star” will still be to get their polling results as close to the actual outcome as possible. Though voters may not always trust the polls — and often dislike what they show — pollsters say they are focused on that target, not on making people happy.

“We’re immune to it by now,” Dr. Paleologos said. “Anyone starting out in the business really needs to have a thick skin.”