EnFoBench: A community-driven energy forecasting benchmark

Forecasting plays a crucial role in the energy domain, including long-term investment planning, and short-term operational optimization. However, there is little clarity as to which algorithms perform best in which conditions, with different studies reporting different, often conflicting, results. Researchers have attempted to address this by relying on forecasting competitions. However, despite their increasing prevalence, forecasting competitions often face challenges in generalizing beyond the competition settings. They also struggle with properly evaluating model performance due to limited datasets and forecast horizons, along with focusing on pre-selected error metrics. In this paper, we introduce EnFoBench, a novel open-source benchmarking toolkit designed to address these challenges and facilitate comprehensive evaluation of energy forecasting models in a transparent and reproducible manner. EnFoBench provides a dynamic framework to benchmark energy forecasting models across various scenarios, by providing homogenized training datasets for building energy demand and generation, and a continuously updated dashboard showcasing how different models perform on a wide variety of metrics for unseen test datasets. This evaluation provides more holistic insights into model performance, ranging from relative forecasting skill compared to baseline models as well as the computational costs of different models.