Predicting Australian energy demand variability using weather data and machine learning

Abstract Managing energy systems requires understanding the variability of energy demand, which on daily timescales is driven primarily by the weather. Historical records of demand typically cover 1–2 decades which may be too short to capture the range of possible demand, particularly for a climate with high interannual variability such as that of Australia. Predicting demand using long records of weather data opens the possibility of more robustly estimating true demand variability. We estimate daily energy demand between 2010 and 2019 for Australian states in the National Electricity Market using machine learning with reanalysis weather variables as predictors. We assess the performance of these models and examine their behaviour to identify which weather variables are most important for predicting demand. We then use the models to estimate demand for the period 1959–2022. We use this 64-year record to quantify how the probability of high demand days can change compared to individual 10-year periods and when conditioned by the phase of the El Ni <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mrow> <mml:mrow> <mml:mover> <mml:mrow> <mml:mi mathvariant="normal">n</mml:mi> </mml:mrow> <mml:mo stretchy="true">~</mml:mo> </mml:mover> </mml:mrow> </mml:mrow> </mml:math> o Southern Oscillation (ENSO). Energy demand can be accurately predicted with weather, with median errors of 2%–4% on years omitted from the training. We show that the probability of extreme demand over different 10-year periods can vary from half to twice as likely, depending on the decade. When further conditioned on ENSO phase, the probabilities can be up to 7 times higher than when using the 64-year period, implying a risk of overestimating weather-related energy demand if shorter records are used. We conclude that machine learning methods can accurately predict energy demand using only weather data, enabling us to estimate demand variability over longer time horizons than is possible with demand observations. These longer records are important when attempting to quantify tail risks of demand, and so can help to inform the design of energy systems.