The accuracy of Sportsbet betting markets in the 2022 Australian federal election

Betting markets are often said to provide more accurate forecasts of elections than polls, despite evidence to the contrary. We contrast the trajectory of Sportsbet’s implied probability that either party would win the 2022 Australian federal election and the performance of Sportsbet’s markets in all 151 House of Representatives seats.

Author
Affiliation
Professor Simon Jackman

University of Sydney

Published

12:43PM 25 November 2022

Summary

  • Unlike 2019, at the end of the 2022 election campaign Sportsbet’s market did accurately predict the election outcome.

  • Sportsbet’s headline betting market on “party to form the next government” exhibited considerable volatility over the election campaign, — even briefly tipping a Coalition re-election at one point, but otherwise suggesting a Labor win — with an average implied probability of around 67% (Section 1). Volatility in the Sportsbet national market was greater than that observed in poll averages (Section 2).

  • Across Sportsbet’s markets for all 151 House of Representatives seats, 16 market favourites were not elected, an accuracy rate of 89.4% (Section 3.1).

  • Sportsbet markets did not predict Greens winning 3 seats in metropolitan Brisbane and only tipped one of the seven gains by independent candidates, over-estimating the Coalition’s seat tally by 7 seats (65 versus 58, Section 3.2).

  • Sportsbet prices in individual seats implied that 11 seats would change hands; 8 did so, with one of those seats (Brisbane) not won by the Sportsbet favourite. Sportsbet prices had an accuracy rate of 128/140 (91.4%) in seats where incumbent candidates/parties were market favourites (Section 3.3). Of the 20 seats changing hands, Sportsbet prices correctly tipped the winner in 7 seats.

  • A “null” model that no seats would change hands would have thus made 20 errors with an accuracy rate of 86.8%. Sportsbet’s markets improved on the accuracy of this null model by a net 4 seats or 2.6 percentage points of accuracy.

  • Survey-based estimates of each seat’s result outperformed the betting markets. YouGov’s MRP estimates — published nine days before the election — produced 12 incorrect calls for an accuracy rate of 92.1%, with 4 fewer seats incorrectly predicted than by Sportsbet at the very end of the campaign (Section 3.4).

  • As is typical for odds set by bookmakers, Sportsbet prices exhibit considerable long-shot bias, over-stating the probabilities of extremely unlikely outcomes and hence under-stating the probabilities of likely outcomes (Section 4.2).

Introduction

We examine the predictive accuracy of Sportsbet’s betting markets in all 151 House of Representatives seats for the 2022 Australian federal election.

We convert Sportsbet closing prices to an implied probability of winning (IPOW) for each candidate, after first excluding long-shot candidates; these procedures are detailed in the analysis of the betting markets we generated during the 2022 campaign.

1 Party to form government

Over the 60 days prior to the election, Sportsbet’s headline “party to form next government” market had Labor always as favourite (see Figure 1), save for a few hours on 18 April 2022 when the Coalition briefly was priced at 1.87 to Labor’s 1.90, corresponding to an IPOW for Labor of 49.6%. Labor’s shortest odds were recorded on 15 May at 1.25, corresponding to an IPOW of 77.1%. Labor’s average IPOW over the 60 day period was 66.2%.

Figure 1: Labor and Coalition implied probabilities of winning in Sportsbet’s headline “party to form next government” betting market, 20 March to 21 May 2022. IPOWs renormalised to sum to 100% after removing long-shot “Other” outcome. Roll over each line to display prices.

2 Betting markets vs polls

Figure 2 shows a small trend towards the Coalition in national polls fielded during the same sixty day period. Almost every poll fielded in this period had the Coalition’s two-party preferred result well below 50% and the resulting poll average suggested the Coalition was performing two to three points below its 2019 polls.

Figure 2: Poll average, Coalition two-party preferred, 20 March to 21 May 2022, computed using methods described in Mansillo and Jackman (2020) and Jackman (2009, 2005). Points indicate polls published during the 2022 campaign; horizontal lines cover field period and publication date; roll over each point to show information. The 2019 poll average is also displayed (time shifted so as to correspond to the 2022 campaign period), computed using the same assumptions used in computing the 2022 poll average, along with the 2019 result.

Poll based estimates of the Coalition’s two-party preferred vote were biased downwards in 2019, by almost three percentage points. Even if 2022 polls were carrying the unusually large degree of bias seen in 2019, the 2022 polls strongly suggested that the Coalition was performing well below its 2019 result, almost surely on track for two-party preferred result well below 50% and quite likely below 49% (the outcome in the 1998 election when the Coalition was nonetheless able to form government).

In short, the polls strongly and consistently indicated that the Coalition would not form government after the election. Compared to the trajectory of the betting markets, a poll-based forecast in 2022 was reasonably accurate (missing the actual two-party preferred result by about a percentage point) and consistent over the campaign period.

In Jackman (2015) I provide a detailed analysis of the interaction between betting markets and polls in the lead-up to the 2013 Australian federal election.

3 House of Representatives seats

Table 1 presents an assessment of the aggregate performance of Sportsbet seat markets, examining counts of seats won against expected counts based on IPOWS. Three counts are presented:

  • using raw, unadjusted IPOWs and the fact that the expected count of a set of uncertain events is equal to the sum of the probabilities of each event. The totals for each party shown in the “Expected seats” column of Table 1 are simply the sum of each party’s IPOWs over the 151 House of Representative seat markets.

  • again summing probabilities over seats by party, but setting IPOWS < 10% to zero and renormalising the remaining non-zero probabilities to sum to one within each division.

  • The third column counts seats where the indicated party has the highest IPOW (or, equivalently, has the shortest odds or prices and is the market favourite in a given seat).

Sportsbet’s markets substantially over-estimated the Coalition’s performance, with the Coalition ending the campaign as favourite in 65 seats; the Coalition won seven fewer. Independents ending the campaign as favourite in only four seats, despite going on to win ten seats. Similarly, Greens candidates ended the campaign as favourites in just one seat but won four.

The extent of longshot bias is apparent when summing across all 151 seats. For instance, UAP and PHON IPOWs should have translated into at least 1 House seat for each party, but none of the 151 UAP House of Representatives candidates won more than 10% of first preferences. Similarly, no PHON candidate won more than exceed 15% of first preferences and none were elected, despite Sportsbet IPOWs indicating an expected haul of 1.5 seats.

3.1 Predictions versus outcomes

In 16 divisions the Sportsbet market favourite was not elected; these appear on the off-diagonal elements of the classification table in Figure 3. 55 of the Coalition’s 58 seat wins were correctly anticipated by the Sportsbet market, as were 73 of Labor’s 77 seats.

Figure 3: Cross-tabulation of party as Sportsbet favourite by party winning each of 151 House of Representatives seats. 16 seats not won by the Sportsbet favourite are in the highlighted off-diagonal cells; roll over the cells to display division names.

3.2 Loses by betting market favourites

Table 2 provides additional detail for the 16 seats where the Sportsbet favourite was not elected. These are a mix of 13 seats unexpectedly changing hands and 3 Coalition retentions:

  • the three Green wins in Brisbane, Griffith and Ryan; Brisbane produced the biggest surprise for the betting markets, the Green candidate winning the seat from 3rd place on 1st preferences and ending the campaign as a long-shot (IPOW of 9.6%).

  • five of the six wins of so-called “teal” independents: Wentworth, Mackellar, North Sydney, Kooyong and Curtin (Zoe Daniel ended the campaign as the Sportsbet favourite in Goldstein).

  • the Coalition retaining the two Tasmanian seats of Bass and Braddon and Longman (QLD).

  • the large swings to Labor in Western Australia, with Sportsbet prices incorrectly tipping the Coalition to retain Tangney and Hasluck.

  • the ALP losing Fowler (NSW) to independent Dai Le.

  • Coalition seats such as Higgins (VIC) and Bennelong (NSW) falling to Labor.

3.3 Predicting seat turnover

Table 3 summarises the accuracy of Sportsbet seat-level predictions, with respect to whether the seat would change hands or the incumbent candidate or party would be re-elected.

Incumbent candidates and/or parties lost in 20 seats. Sportsbet’s markets anticipated 8 of these loses (but not the winning party in Brisbane). In three seats incumbents candidates and/or parties were returned when Sportsbet markets anticipated the seat changing hands: two of these seats, Bass and Braddon are in Tasmania and the other is Longman (QLD). Put differently, 8 out of Sportsbet’s 11 (72.8%) predictions that incumbent parties or candidates would lose were correct; 128 of 140 (91.4%) predictions that incumbent candidates/parties would be returned were correct.

3.4 Comparison with survey-based predictions

YouGov Australia used a large survey and multivariate regression and post-stratification (or MRP) to produce estimates of vote shares in each House of Representatives division. These estimates were published by The Australian newspaper on 12 May 2022, nine days prior to Election Day. YouGov also produced estimates of two-candidate preferred vote shares, and hence forecasts of winners of each seat.

YouGov’s MRP estimates outperformed betting markets by a net 4 seats correctly called, with 12 incorrect predictions, all of which were also incorrectly forecast by Sportsbet markets. The improvements over Sportsbet were to correctly predict Coalition loses in Kooyong (VIC) and Higgins (VIC) and the Coalition’s retentions of Braddon (TAS) and Longman (QLD).

4 Are Sportsbet probabilities well calibrated to actual election results?

Probabilities are well calibrated when events happen with rates equal to their corresponding probabilities. Events assigned low probabilities should occur rarely and conversely high probability events should occur frequently, if the probabilities are well-calibrated with reality. More generally, events with probability in a neighbourhood of \(p\) should occur \(p \times\ 100\) percent of the time. Calibration is thus a very desirable property for any probabilistic forecast.

We assess the calibration of Sportsbet IPOWs in House of Representative seats in the 2022 election in two ways:

  • assessing the relationship between IPOWs and two-candidate preferred vote shares for candidates and parties in the “final two” in a division (Section 4.1)

  • assessing the relationship between IPOWs and rates of winning (Section 4.2)

4.1 Relationship between IPOWs and TCP

Figure 4: Two-candidate preferred vote share plotted against Sportsbet closing IPOW. Each data point is a House of Representatives division; highlighted points in the off-diagnoals quadrants are seats where the Sportsbet favourite did not win. Orange lines indicate trends. Roll over each data point to display division name and other information.

4.2 Calibration of forecasts and long-shot biases

In Figure 5 we assess the calibration of Sportsbet IPOWs; for this analysis we use IPOWs without any adjustment for long-shot bias, investigating the magnitude of these biases in Sportsbet prices.

Figure 5: Calibration curves for Sportsbet IPOWs (unadjusted for long shot bias, horizontal axis). The actual proportion of wins is plotted on the vertical axis, shown with the orange line (estimated by locally linear regression); for well-calibrated probabilistic forecasts this line should not differ markedly from the black, 45 degree line. The local linear regressions of outcomes on IPOWs are fit using loess with bandwidth selected so as to minimise cross-validation error.

Again we note the overly bullish Coalition IPOWs in Sportsbet markets. The Coalition won no seats where its IPOW was below 44%, reflected in the calibration curve not moving from zero until IPOW ≅ 20%. Betting markets also overstated the Coalition’s success in “toss-up” seats with IPOWs close to 50%, with the calibration curve placing the Coalition win rate at 42%.

Conversely, betting markets underestimated Labor performance slightly in toss-up seats, but markedly in seats where Labor was the market favourite. Labor won 75% of those seats with IPOW in the neighbourhood of 60% and every seat where its IPOW exceeded 65%.

As expected, longshot bias is clearly apparent in Figure 5. Sportsbet, like many betting agencies, attempts to guarantee its profits by offering prices that are too optimistic for long shots, seeking to entice wagers on outcomes that highly unlikely and fewer wagers on likely outcome. As a consequence, this deflates the IPOW of “sure things” and inflates the IPOW of virtually impossible outcomes.

5 Movement over the campaign

References

Jackman, Simon. 2005. “Pooling the Polls over an Election Campaign.” Australian Journal of Political Science 40 (4): 499–517. https://doi.org/10.1080/10361140500302472.
———. 2009. Bayesian Analysis for the Social Sciences. Wiley Series in Probability and Statistics. Chichester, U.K: Wiley.
———. 2015. “All That Glitters: Betting Markets and the 2013 Australian Federal Election.” In Abbott’s Gambit : The 2013 Australian Federal Election, edited by Carol Johnson, Wanna, John, and Lee, Hsu-Ann. Canberra: ANU Press. https://doi.org/10.22459/AG.01.2015.09.
Mansillo, Luke, and Simon Jackman. 2020. “National Polling and Other Disasters.” In Morrison’s Miracle: The 2019 Australian Federal Election, edited by Marian Simms, Anika Gauja, and Marian Sawer. Canberra: ANU Press. https://doi.org/10.22459/MM.2020.07.