Precision powered: revolutionize energy consumption forecasting using transformers

The enormous and increasing impacts of urbanization and demographic shifts in Pakistan on energy demand are causing huge obstacles for energy management which is happening at a time when we are facing an acute energy crisis. The demand-supply gap is widening, necessitating the development of state-of-the-art models for forecasting energy consumption using machine learning (ML) and deep learning (DL). Previous models like Autoregressive Integrated Moving Average (ARIMA) and Seasonal Autoregressive Integrated Moving Average (SARIMA) helped to interpret consumption behavior insights. The major contribution of this paper is proposing and applying the new Sequential Transformer model which significantly provides an improvement over traditional models such as ANN, Vanilla Transformer, and Long short-term Memory (LSTM) giving prediction performance with better accuracy having lower computational cost. Our proposed model has a lower mean squared error (MSE), converges faster, and is more robust in predicting electricity consumption through extensive evaluations on two benchmark household electricity consumption datasets. In particular, our Sequential Transformer reaches MSE values of 0.012 and 0.115 on the two datasets, respectively, outperforming best alternative models by a large margin. This is a significant development that will help alleviate Pakistan’s energy crisis as it allows for accurate and real-time strategies for managing electricity potentially leading to timely decisions on energy generation based on consumption forecasting. Index Terms—Energy Consumption, Transformer, Deep Learning, Long short-term memory (LSTM), Predictive Modeling, Autoregressive Integrated Moving Average (ARIMA), Seasonal Autoregressive Integrated Moving Average (SARIMA).