What is the probability Harris wins? Building a Statistical Model.

After Joe Biden made the historic decision to exit the presidential race, the race entered a new exciting time with lots of uncertain questions. In this article we'll try to see just how uncertain we are, and build a probabilistic model of whether Kamala Harris will win.

There are lots of online election forecasts (e.g., from Silver et al., Morris et al., Gelman et al., among others). So why build another one? In a previous article I created an election model to help understand if Biden should drop out (focusing on Gretchen Whitmer as a replacement). That had a motivation and rhetorical purpose. I partially made a version for Harris, but didn't publish before Biden dropped out. So while this Harris model is now mostly recreational, it has some potential contributions for those interested in election modeling:

• Transparency - Each part tries to include links to corresponding source code
• Analysis and visualization of the amount of available polling (Section 1.2). This is to model the potentially limited initial Harris polling.
• Alternative approach/presentation of errors and poll movement (Section 2)
• Timeliness/impatience - At time of writing, prominent modelers like FiveThirtyEight, Silver, and the Economist have/had not released a version for Harris yet. This gives one initial estimate.

For those just looking for a number, here's jumping ahead for the top-line:

This estimate should automatically update several times a day with the newest polling data. For reference, here's archived past versions.

We'll go through each of the steps to reach this estimate. The general approach follows that of other polls-based models. We create an average of polls in each swing state, estimate the expected polling miss, and then estimate how polls might move. Then we run thousands of random simulationsto get a fractionwhere each candidate wins.

Our modeling focuses on the 8 swing states that are most likely to decide the election (ie, be the "tipping point").

Where are Polls Today

Gathering Data

The first step is gathering polling data (sourced via FiveThirtyEight).

Not all pollsters are equal, with some pollsters having a better track record. Thus, we weight each poll. Our weighting is intended to be scaled where 1.0 is the value of a poll from a top-rated pollster (eg, Siena/NYT, Emerson College, Marquette University, etc.) that interviewed their sample yesterday or sooner.

Less reliable/transparent pollsters are weighted as some fraction of 1.0. Additionally, older polls are weighted lessWeight decays with an approximate half-life of 9 days. We use between the start and end date, and wait 1.5 days to start decay.. Polls before Biden dropped out carry ¼ weight. This function is only a heuristic estimate (see codefor exact definition).

If a pollster reports multiple numbers (eg, with or without RFK Jr., registered voters or likely voters, etc), we use the version with the largest sum covered by the Democrat and Republican.

Table Source

Estimating Poll Miss

Morris (2024) at FiveThirtyEight reports that the polling average typically misses the actual swing state result by about ~2 points for a given candidate (or ~3.8 points for the margin). This is pretty remarkable. Even combining dozens of pollsters each asking thousands of people their vote right before the election, we still expect to be several points off. Elections are hard to predict.

Our current situation is even more uncertain. The ~2 points of miss is for a typical candidate on election day. With Harris we start in a time with potentially limited polling data. To estimate this we look at the weighted count of polls.

Right now we estimate we have the equivalent of 16.9 top-quality national polls for Harris. For comparison, we estimate we had 22.1 top-quality national polls for Biden the day before he dropped out and 59.5 top-quality polls for Biden on 2020 Election Day.

For swing state polls we apply the same weighting. To fill in gaps in swing state polling, we also combine with national polling. Each state has a different relationship to national polls. We fit a linear function (ie, a slope and an intercept) going from our custom national polling average to 538's state polling average for Biden in 2020 and 2024. We average this mapped value with available polls (its weight is somewhat arbitrarily defined as the $R^2$ of the linear fit). We highlight that the national polling-average was highly predictive of 538's swing state polling-averages (avg $R^2=0.91$).

In Figure 1 we show both the weighted count of polls and the square root of the weighted count. Probability theory tells us that sampling is not linear. As an example, say we had a poll of 1000 people estimating a vote of 45% with some amount of error. If we repeated that poll and now had 2000 people, we would not be twice as confident of our estimate. Instead, we would need roughly 4000 people to be twice as confident. Having 16.9 polls now compared to 59.5 polls on election day 2020 is like having $\sqrt{16.9}/\sqrt{59.5}\approx 0.5$ as much polling information.

We make the assumption that the amount of swing state polls we had on election day 2020 was enough for the typical ~2 point average miss. We then estimate expected error in each swing state given how many polls we currently have there. We assume that only half of the average miss (ie, ~1 point) could be reduced with more polling. The other half this is half fraction is purely heuristic. More rigorous work could try to estimate this empirically of the miss is some unrecoverable error (for example due inability to contact subsets of voters, or industry-wide methodology flaws). See codefor precise details.

Using this method, we estimate that the average swing state polling miss is currently 3.0 points.

We emphasize the average miss here is just an average across thousands of simulations. The actual miss can be higher or lower in either direction. Following Morris (2024), we model the distribution of these errors as a t-distribution with five degrees of freedom.

Additionally, we assume poll misses are correlatedbetween states. Similar to our previous Whitmer model, we use correlations use from the 2020 FiveThirtyEight and Economist models (via Pearce (2020)). A more pure version of this model would try to reestimate this from data.

Estimates with Poll Miss

If we use this expected miss and pretend the election was today, we would estimate a 57% chance Harris would win.

Where will Polls be in 57 Days

If a candidate is behind, they would hope for more variance in outcomes. If they are ahead, less variance is better. In addition to typical poll misses, we would expect some variance from movement polls in the next 57 days to Election Day.

Here we show the trends in 2020 and 2024. Dem Share is share of vote with just the Democrats and Republicans, ie $(Dem/(Dem+Rep))·100$. Keep in mind, the magnitude of shifts in this value is half that of shifts in margin.

The average 57-day movement in 2020 was 0.79 points and the average 57-day movement in Biden 2024 was 0.55 points (movements are absolute value. It could be up or down.). The largest movement observed for Biden is 3.4 points in Florida between May 26, 2020 and July 22, 2020. During this time Covid's impact grew to over 600,000 Americans dead while Trump mused about injecting people with disinfectants and slowing testing to make himself look better.

We attempt to estimate the mean expected move for Harris, just based on limited polling so far. We do this via a rough process of random walks sampled from her movement so far (src). We then have an estimate of an average expected move of 5.03 using just Harris 2024 data.

To get a final estimate of Harris's expected move in each state, we average together Biden 2020, Biden 2024, and Harris 2024 and blend data for a given state and the national moves. Please refer to the codefor a precise definition. This process estimates an average expected move across the 8 swing states is 2.13 points using all data.

Using this expected average, we modelthe distribution of movements as a t-distribution with 5 degrees of freedom.

Results with Movement

Here show estimated win probability taking into account both the average 3.0 point polling miss and the expected 2.13 point average poll movement.

Thus, allowing variance from movement slightly changes odds for Harris. One might intuitively expect a larger change, however we must remember that under this model, the polling miss and poll movement is assumed to be independent. As an example, we could sample a 2 point move up in polls, but this could be cancelled out by sampling a -3 point poll miss. The average combined poll miss is 3.8 points.

Model Limitations

There are several limitations of this model

Polling is a limited tool: As mentioned earlier, even with data from dozens of pollsters right before Election Day, polls typically miss by several points. One thing I took away from building this model is an encouragement to just care about polls less. We are unlikely to end up in a world on election day where polls tell us much different than a coinflip.

No mean-reversion or trends: This model assumes misses in either direction equally probable. However, in reality we should expect moves far from the historical mean to be less likely (as these voters become increasingly partisan).

Poll uncertainty quantification not empirically validated: We make assumptions about the behavior of poll uncertainty for a given number of polls that might not be valid (in particular the fraction of aleatoric uncertainty is unclear). As mentioned, stronger work would better estimate this using data from past races.

Not all states: We only model 8 swing states. This is because in situations where a state like Texas or Iowa goes blue, the election is almost certainly already decided elsewhere. We also don't model the atypical way Nebraska and Maine distribute their electors.

Ignoring 3rd party votes: Better factoring in RFK Jr. might slightly change things. Also, undecided voters are essentially assumed to split equally rather than more complex schemes.

Correlations simple: This model is not completely pure, as it uses state correlations extracted from other models. Additionally, we only consider correlations at a state-level correlations, rather than more complex schemes.

Many assumptions of this model are rough and not particularly principled. However, at least so far estimates from this model have mostly aligned with those of far more complex models.

Conclusion

The race is highly uncertain. Considering both the "Election Today" results with poll misses and the poll movement estimates, the race is close to a coin flip. Making confident predictions either way is unwise.

As a reference, using polling data from July 21 this model would have estimated that Biden had a 27% chance of winning when he dropped out with 107 days to the election. Thus, with Harris estimated at 57% today, we'd estimate she so far has improved odds over Biden then. There are potentially exciting times ahead.