Non-parametric testing of the machine learning electricity prices forecasts

This research analyzes forecast accuracy in the day-ahead electricity market. Performance of Random Forest and XGBoost machine learning models is compared based on the day-ahead electricity market data for Germany. Data for 2018 and 2021 is analyzed in order to explore differences in forecast accuracy in the low and high market volatility periods. Initial training data for 2017 is used in order to produce forecasts for 2018 up to one month ahead. The training set is then rolled one month forward thus creating a fixed length rolling window of training and forecast set data for the remainder of the analyzed period. This methodological framework results in 11 forecasting sets for each analyzed year. Forecast accuracy is then evaluated by comparing root-mean-squared errors (RMSE) for the observed period. The focus of the research is on examination whether differences in the RMSE values of the competing machine learning models being analyzed can be reliably determined. For this purpose, firstly forecasting exercise has been conducted 30 times over for both machine learning models and each forecast set containing all forecast horizons. Secondly, median RMSE values are analyzed for each forecast set and non-parametric Wilcoxon rank-sum test is used to determine whether the observed differences in RMSE are statistically significant. Research results show small differences in RMSE values, however, they are found to be statistically significant for all forecast sets except one. Moreover, Random Forest seems to slightly outperform XGBoost model during the period of low market volatility, while XGBoost seems to perform better in the last three forecast sets of 2021 associated with higher market volatility.