It’s election season. A time when the fancy of political prognosticators turns to opinion polling. Polls are one of the only objective measures of how a race is looking before the official votes are cast, but there is always a question of whether polling is reliable.
Especially since Donald Trump’s unexpected victory in 2016, many consider polls unreliable. The truth is that the polls in 2016 weren’t off as far as many think they were. The popular vote corresponded almost exactly to the national polls in which Hillary Clinton led by about two percentage points. At the state level, Trump eked out victories in a number of states where Hillary led in the polls, but polling in most of those states showed a very close race. Wisconsin, where polling showed Hillary up by about eight points, is the exception. Data from the American Association for Public Opinion Research shows that much of the error was due to a final-week surge after James Comey released his letter to Congress on Oct. 28, 2016.
As I have written before, polls should neither be believed absolutely nor totally discarded. The best strategy is to take each individual poll with a grain of salt. Rather than focusing on a single poll, look for trends. This is easy to do with sites such as Real Clear Politics and FiveThirtyEight that provide comprehensive lists of polls as well as averaging their results. Another good strategy is to discard outliers, single polls that show a radically different result from the rest.
The best way to judge the accuracy of polls is to compare them to the final election results. Since polls are snapshots that record a moment in time, rather than forecasts, this can be done by looking at the polls taken just before the election. Fortunately, RCP and FiveThirtyEight both keep their polling data posted for past elections.
If we compare polling for the four special elections held so far this year with the actual election results, this is what we find:
Pennsylvania’s 18th district was the first special election held this year. It pitted Democrat Conor Lamb against Republican Rick Saccone for the seat of Tim Murphy, a Republican who resigned due to a sex scandal. RCP showed Lamb leading by two to four points in two of the last three polls, giving him an average advantage of two points. When the results were in, Lamb won by 0.4 points.
Next, Hiral Tipimeni (D) and Debbie Lesko (R) vied for the seat of Trent Franks, a Republican who resigned after being accused of sexual harassment for asking to impregnate female staffers. For this race in Arizona’s sixth district, RCP shows two polls. Lesko led in both by an average of eight points. On Election Day, Lesko won by 5.2 points. The 2.8-point difference is the largest error of the three elections.
In the third special election, the race for Texas’ 27th district, there seem have been no public polls. This is not surprising since the district is reliably Republican and was carried by Donald Trump by almost 20 points.
The fourth election was in Ohio’s 12th district where Republican Pat Tiberi resigned to lead the American Business Roundtable. Democrat Danny O’Connor and Republican Troy Balderson ran to fill the seat in this closely watched race. Polling in the race was mixed, especially in the final days. The last two polls showed each candidate up by one point and the RCP average was a tie. Balderson won by 0.8 points.
While polls aren’t expected to be exact or forecast the exact outcome on Election Day, special election polling in 2018 has been pretty good. Over three separate races, the average error for the average of polls was only 1.7 percentage points. Considering that the polls in some races were taken weeks before the election and late-breaking events, such as President Trump’s visit to Ohio three days before the election, can cause voters to change their minds, the results are remarkably accurate.
Don’t assume that every poll that you see is gospel, but if you want an idea of how elections are going to play out, there is no substitute for a close look at the polls. The bottom line is that if candidates didn’t think polls were valuable tools, they wouldn’t pay for them.