A hybrid deep learning framework for short-term load forecasting with improved data cleansing and preprocessing techniques

• A novel preprocessing technique with O(N) time complexity is proposed which is 87 % more accurate than the standard imputation methods. • A simple hybrid deep learning model for STLF is proposed having fewer parameters and less training and testing time. • The accuracy of the STLF model is enhanced by reducing the MSE, RMSE, MAE, and MAPE to 0.0054, 0.0734, 0.0568, and 0.2424 respectively. • A conceptual framework is proposed to enhance the grid's reliability by using the forecasted generation and demand values. The dynamic variations in connected loads and the intermittent nature of renewable energy sources significantly impact smart grid reliability. Accurate load and generation forecasting stand pivotal for enhancing grid reliability and efficient operations. In this study, a comprehensive four-step approach is introduced for short-term load forecasting (STLF) aimed at precisely estimating power demand and generation. The process unfolds with data collection, followed by rigorous standardization, preprocessing, and cleansing of demand and generation data. Subsequently, a hybrid deep learning model, comprising bidirectional long short-term memory (BiLSTM), bidirectional gated recurrent unit (BiGRU), and a fully connected layer is trained using the clean data. This model harnesses the temporal dependencies within the data for accurate predictions. The model's performance is then evaluated using mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE), providing forecasted values for both generation and demand on minute, hourly, daily, and weekly intervals. Notably, the proposed approach achieves a remarkable MSE of 0.0058 for load forecasting and 0.0033 for generation forecasting. Comparative analysis with state-of-the-art (SOTA) techniques in terms of accuracy and computational cost underscores the superior accuracy of the proposed framework in forecasting both generation and demand. Importantly, the proposed approach bridges the gap in reliability enhancement for smart grids operating, a facet lacking in many existing methodologies. This signifies the potential of the proposed approach to bolster smart grid reliability, ensuring more reliable operations.