Dual-stream representation learning with large language models for electricity demand forecasting

Electricity demand forecasting is crucial for the sustainable management of electricity markets and the reliable operation of power systems, particularly in the context of rapidly urbanizing and energy-intensive cities. Accurate load prediction supports efficient energy scheduling, reduces operational inefficiencies, and contributes to lower carbon emissions by enabling better integration of clean and distributed energy resources. Given the rise in electricity consumption corresponding to population growth, precision and accuracy in predicting electricity demand over time are increasingly significant for achieving energy-efficient and resilient urban infrastructures. In this direction, the current study presents a novel hybrid framework that integrates advanced data pre-processing, dual-path feature extraction, and large language transformer modeling to enhance the accuracy of load predictions in smart grid environments. The methodology begins with gaussian smoothing to mitigate noise and rectify inconsistencies in raw consumption data. A dual-stream module simultaneously analyses time-series patches while calculating key statistical summaries, producing robust patch embeddings and distinct prompt tokens. The prompt tokens act as contextual signals for a partially fine-tuned Generative Pre-trained Transformer (GPT-2), while a prototype-based patch reprogramming module continuously enhances the patch embeddings by aligning them with a collection of learnable prototype vectors. This design allows the transformer’s self-attention mechanism to effectively capture both local fluctuations and long-term temporal dependencies present in electricity demand. Thorough experiments conducted on a curated regional dataset illustrating electricity load patterns across the four southern states of India reveal that the proposed method outperforms traditional deep learning and hybrid techniques, achieving an average Root Mean Squared Error (RMSE) of 1.054 and a Mean Absolute Error (MAE) of 0.8579. The proposed framework demonstrates the desired generalization capacity with parameters efficient adaptation of large language models for electricity forecasting, thereby making it suitable for load generation, management, and planning activities. The findings offer new perspectives on how data-driven methods might be combined with large language models to improve energy forecasting and achieve sustainability.