Poll averages for the 2022 Australian election
Professor Simon Jackman
23 November 2022
- This document was updated regularly over the course of the 2022 election campaign; with actual 2022 election results now in hand, this document serves as an archive of sorts, a basis for examining the performance of the polls and poll averages (to be presented elsewhere). With just one or two exceptions highlighted in red, this document is as it was Election Day 21 May 2022.
1 Summary
Averages of recent polls indicate the Coalition will win 46.5% of the two-party preferred (TPP) vote; see Table 1.1.
The Coalition’s estimated first preference vote of 35.8% is 5.6 percentage points down on 2019; Labor’s first preference share is estimated at 35.8%, up 2.5 points from 2019 and the Greens are estimated to have 12.0% of first preferences, up 1.6 points from 2019.
But if public, national polls are carrying the error they exhibited in 2019, the election is closer to 50.8-49.2 ALP/Coalition (TPP).
If the polls are carrying the average level of bias seen in the five previous elections (2007 to 2019), then the election is more likely 52.1% ALP TPP to the Coalition’s 47.9%.
UPDATE (post-election): these average-bias-adjusted TPP results match the actual election TPP results.
It is extremely unlikely that the Coalition can form government if it were to win 47% or less of the national TPP vote.
The Coalition did form government in 1998 with 49% of the national TPP.
At the end of the campaign:
| ALP | LNP | GRN | LNP_2PP | |
|---|---|---|---|---|
| Unadjusted poll average | 35.8 | 35.3 | 12.0 | 46.7 |
| Correct for 2019 error | 34.2 | 38.9 | 11.9 | 49.2 |
| Correct for average error | 35.2 | 36.8 | 11.0 | 47.9 |
| Integrating uncertainty in average & error | 35.7 | 37.1 | 11.1 | 47.6 |
| 2019 results | 33.3 | 41.4 | 10.4 | 51.5 |
| 2022 results | 32.6 | 35.7 | 12.2 | 47.9 |
2 Errors in poll averages, Australian elections 2007 to 2019
Averages of opinion polls figure prominently in commentary and prognostication about the upcoming Federal election. My work is being used by the ABC regularly, including on its flagship Insiders programme.
By and large, 2019 pre-election polls — and averages of those polls — missed the mark. After vigorous introspection, and the formation of an Australian Polling Council, some polling firms have changed their methodology and methods of reporting. Polls fielded shortly before recent state elections were reasonably accurate, bolstering confidence that 2022 election polls will be more accurate than in 2019.
We draw on analysis presented in summary form in Mansillo and Jackman (2020), where a poll-averaging model was fit to Australian opinion polls fielded in the period 2007-2019.
Details on the poll averaging model and fitting it to published polling data appear in the Appendix.
Figure 2.1 shows the performance of poll averages in five previous elections: 2007, 2010, 2013, 2016 and 2019, graphing the error of the poll average computed over time until the election. Error is defined as poll average minus the actual election result, so that
positive errors mean the polls are over-estimates of a party’s actual level of support, or that the polls are collectively biased upwards with respect to that party
conversely, negative errors mean the polls are under-estimates of a party’s actual level of support, or that the polls are collectively biased downwards with respect to that party
Since the poll average can be updated daily (even if the absence of new polls), we compute and plot errors in the poll average as daily time series in Figure 2.1. 2019 poll error is highlighted in orange, while the blue line displays the average of the poll errors on a particular day for each party, averaging over the five elections (2007 to 2019, inclusive). The graph zooms in on the sixty days prior to each election.
Figure 2.1: Errors in poll averages, by party, year and days until election, 2007, 2010, 2013, 2016 and 2019.
First, the good news:
as the election gets closer, poll averages perform better, with errors getting smaller, with the possible exception of estimates of vote shares for the Greens.
by Election Day, the average error of the poll averages is generally small, ranging from 0.5 for the Labor Party’s primary vote share to -1.3 for the Coalition’s share of the two-party preferred vote; see Table 2.1.
the party winning the two-party preferred vote almost always forms government: rare exceptions are 1998 and 1990. Between 2007 and 2019, poll averages of the two-party preferred result have picked the election winner correctly, save in 2019. In 2010 the polls under-estimated the Coalition’s performance by a sizeable margin with a poll average of 48.1% versus the actual result of 49.9% and 73 of 150 seats.
| year | ALP | LNP | GRN | OTH | LNP2PP | actual |
|---|---|---|---|---|---|---|
| 2007 | 1.7 | -1.0 | 0.4 | -1.4 | -1.6 | 47.3 |
| 2010 | 0.6 | -1.5 | 1.3 | -0.3 | -1.7 | 49.9 |
| 2013 | 0.6 | -1.2 | 1.5 | -0.8 | -0.7 | 53.5 |
| 2016 | -1.6 | 0.1 | 1.3 | -0.1 | 0.6 | 50.4 |
| 2019 | 1.6 | -3.6 | 0.1 | 2.0 | -2.5 | 51.5 |
| Average | 0.6 | -1.4 | 0.9 | -0.1 | -1.2 |
The less good news:
Australian polls exhibit biases that persist over decades. ALP and Green 1st preferences are typically over-estimated, while Coalition and “Other” are usually under-estimated by averaging over public polls.
Unsurprisingly then, poll averages typically underestimate the Coalition’s share of the two-party preferred vote, by an average of 1.1 percentage points.
The poll errors of 2019 are large both in absolute terms and relative to errors in other elections. The underestimate of the Coalition’s first preference vote share, 3.6 percentage points, has a magnitude more than twice that seen in the four previous elections for either major party. The 2.5 percentage point miss on the two-party preferred vote in 2019 was more than twice the average magnitude of the corresponding errors in the earlier elections.
3 Poll averages for 2022: the story so far
What does this past history with respect to the errors in poll averages imply for the poll averages constructed from the 2022 data?
We begin by presenting poll averages from the 2019-2022 election cycle, to date.
Figure 3.1: ALP 1st preferences
Figure 3.2: LNP 1st preferences
Figure 3.3: GRN 1st preferences
Figure 3.4: LNP two party preferred
3.1 2022 zoom-in with 2019 overlay
We also zoom-in on calendar year 2022, overlaying the poll average from the same stage of the 2019 campaign.
Figure 3.5: ALP 1st preferences, 2022
Figure 3.6: LNP 1st preferences, 2022
Figure 3.7: GRN 1st preferences, 2022
Figure 3.8: LNP two party preferred, 2022
4 Adjusting 2022 poll averages given errors in past years
We consider three adjustments to the 2022 poll averages:
day-specific 2019 error.
day-specific average error, computed from poll averages 2007 to 2019, inclusive.
sampling from error distribution given errors listed in Table 2.1, subject to the constraint that there is a 20% chance of errors larger than the largest errors observed in previous elections.
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_ALP_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_ALP_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_ALP_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_LNP_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_LNP_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_LNP_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_GRN_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_GRN_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_GRN_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_LNP_2PP_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_LNP_2PP_2022-07-16_sum_to_zero.rds"
## [1] "/Users/jackman/Documents/Projects/oz/2022/poll_averaging/output/xi_LNP_2PP_2022-07-16_sum_to_zero.rds"
Figure 4.1: Adjusted poll average, ALP 1st preferences
Figure 4.2: Adjusted poll average, Coalition 1st preferences
Figure 4.3: Adjusted poll average, Green 1st preferences
Figure 4.4: Adjusted poll average, LNP two-party preferred
5 Appendix
5.1 Poll average model
Each poll \(p \in 1, \ldots, P\) is fielded on day \(t(p)\) by polling firm (“house”) \(j(p)\).
Each poll for each party (not indexed here for clarity) yields a proportion \(y_p\) and a sample size \(n_p\).
Polls are modelled as \(y_p \sim N(\mu_p, s_p^2)\):
\(\mu_p = \xi_{t(p)} + \delta_{j(p)}\),
\(\xi_t\) the underlying level of support for the party in question on day \(t\).
\(\delta_j\) is a bias term (“house effect”) for polling firm \(j\).
\(s_p^2 = y_p (1 - y_p)/n_p\) is an approximation to the sampling variability of \(y_p\).
Underlying levels of support evolve according to a “locally constant” or Gaussian random walk, \(\xi_t \sim N(\xi_{t-1}, \omega^2)\) where
\(\omega^2\) is the variance of day-to-day innovations in underlying voting intentions
\(\xi_1\) is observed, the level of support for the political party in question on day \(t=1\), the day of the last election.
In summary:
Observed: \(y_p, n_p, \xi_1\).
Unknowns: \(\xi_2, \ldots, \xi_T, \omega^2, \delta_j\).
We impose the normalising assumption \(\sum_j \delta_j = 0\), an assumption that the polls are collectively unbiased.
We approach estimation and inference for the unknown model parameters in a Bayesian setting, using Markov chain Monte Carlo methods to explore the posterior density of the parameters given observables and the model structure.
See Jackman (2009) and Jackman (2005) for additional details.
R and stan code appears in the github
archive
for this project.
5.2 Data
Poll data for 2022 used for this project is listed below. The file R/process_poll_data.R contains code for preparing published poll results for incorporation into the poll-average model and analysis, with adjustments for allocation of undecided percentages, the translation to two-party preferred estimates, missing effective sample sizes and so on.
These data were collected by Casey Briggs and co-workers at the Australian Broadcasting Corporation.