Input-Adaptive Dynamic Convolution-Augmented Transformer for Energy Demand Forecasting

Energy demand forecasting is a critical task in sustainable energy systems, where nonlinearities, temporal dynamics, and multivariate factors pose significant challenges. Despite the success of deep learning methods, how to design and optimize deep learning algorithms is yet to be investigated for energy demand forecasting. To address these issues, we propose an input-adaptive dynamic convolution-augmented Transformer. Specifically, channel, phase, and joint channel–phase embeddings are designed to enrich feature representations. Then, a dynamic convolution module is developed to adaptively learn one-dimensional kernels via input-dependent adaptive attention. The resulting features are further fed into a Transformer encoder for forecasting energy demand. Finally, an efficient Bayesian optimization algorithm is used to automatically design and optimize the proposed method. Experimental results on real-world energy demand forecasting task demonstrate the effectiveness and practicality of the proposed method.