Time series forecasting with variable-centric spectral transformer

Abstract Although Transformers have demonstrated remarkable success in natural language processing, their direct application to time series forecasting faces significant challenges. Traditional Transformers treat tokens as semantic units (e.g. words), whereas tokens in time series (e.g. individual data points) lack semantic coherence, leading to suboptimal modeling of long-term dependencies. Additionally, conventional positional encodings struggle to capture periodic patterns in temporal data, and the quadratic computational complexity of self-attention mechanisms becomes prohibitive for large-scale sequences. We propose ICLformer, the first transformer variant that integrates transposed tokens and complex frequency domain linear interpolation, to address two critical limitations of conventional Transformers in time series forecasting: (1) redefining temporal representations via transposed tokens to model global dependencies across time steps in a variable-centric manner; (2) introducing frequency domain interpolation to preserve periodic features while achieving data imputation and noise reduction with linear computational complexity. Our model excels in accuracy, stability, and computational efficiency compared with traditional methods.