Enhanced very short-term load forecasting with multi-lag feature engineering and prophet-XGBoost-CatBoost architecture

An efficient very short-term load forecast (VSTLF) is essential for power plant unit scheduling because it reduces the planning uncertainty caused by variable renewable energy outputs, which in turn decreases the overall electricity production costs in a power production system. Even though this sector has seen a number of studies and applications in plant scheduling, improving projections with minimum errors is still necessary. In this paper, a novel machine learning framework for predicting very short-term electricity usage was developed. The data utilized in this study are publicly available and were accessed from Kaggle's platform. The framework involved two pivotal stages in its development: robust feature engineering and electric load forecasting via a Prophet-XGBoost-CatBoost (Pr-XGB-CB) model. Feature engineering involves the use of multiple sets of historical electricity consumption data with different time lags for analysis and enhancement. Prophet dissects forecasted data into understandable seasonal, trend, and holiday segments, whereas XGBoost stands out for its speed and effectiveness, especially when dealing with numerous features. To obtain the ensembled forecast, the CatBoost algorithm was utilized. The optimal hyperparameters for the model are evaluated by Optuna. The effectiveness of the suggested model was evaluated by contrasting its performance with that of five alternative machine learning (ML) and deep learning (DL) algorithms. In addition, a sensitivity analysis was conducted to assess the model's robustness under scenarios of missing or limited data, demonstrating its resilience in real-world applications. The proposed framework outperformed the state-of-the-art (SOTA) models, with a Mean Absolute Error (MAE) of 23.70, Root Mean Squared Error (RMSE) of 32.32, and an R 2 of 0.97; hence, the proposed framework performs an efficient guiding role in electrical load prediction. This research offered practical significance by enhancing power plant scheduling efficiency and reducing overall electricity production costs through superior predictive accuracy. • Developed a novel hybrid Pr-XGB-CB model for VST load forecasting. • Employed robust feature engineering with historical lagged data. • Used Optuna for optimal hyperparameter tuning of the ensemble model. • Outperformed 5 SOTA ML/DL models with high accuracy (R 2 = 0.97). • Improved power plant scheduling by reducing load prediction uncertainty.