Feature engineering and selection for prosumer electricity consumption and production forecasting: A comprehensive framework

The integration of prosumers—entities that both consume and produce energy—into the electricity grid marks a pivotal shift towards decentralized and renewable energy systems. While the involvement of prosumers promotes sustainability and energy independence, it also introduces complexities in grid management due to the unpredictable nature of decentralized energy generation. This paper addresses the operational challenges and forecasting inaccuracies arising from the integration of prosumers by leveraging advanced analytical methodologies. Specifically, time series analysis and Fast Fourier Transform are employed to dissect the temporal dynamics and identify key cycles in energy usage and production patterns. Utilizing a dataset provided by Enefit, a leading energy company in the Baltic region, 615 features along with three transformed targets are developed, employing feature generation techniques to enhance the predictive accuracy of models. The study emphasizes the importance of feature selection and the application of ensemble Light Gradient Boosting Machine models to improve forecasting performance. Findings demonstrate that targeted feature selection significantly bolsters forecasting accuracy, with the ensemble model achieving an R 2 of 0.96 and a NMBE of −0.39 for energy consumption and production predictions. Feature selection reduced the number of features by 34.36 % while improving model performance, with NMBE and CV-RMSE decreasing by 9.40 % and 9.25 % respectively. The analysis reveals distinct patterns in electricity consumption and production across different prosumer categories and provides insights into the critical factors influential in forecasting accuracy. This research helps mitigate energy imbalances and sets a foundation for adaptive energy management practices that can be generalized across the energy sector. Through these efforts, the study contributes to the evolution of energy management strategies, paving the way for more reliable, efficient, and sustainable energy systems in the prosumer era. • Framework for feature engineering in prosumer energy forecasting. • Integration of diverse data sources and analytics improves prediction accuracy. • Backward wrapped feature selection tailored to LightGBM enhances performance. • Time series analysis reveals distinct patterns across prosumer categories. • Findings enable precise forecasting to support grid stability and sustainability.