Optimising Time Series Forecasting Transformers to Linear Complexity

ABSTRACT Accurate time series forecasting is a cornerstone of contemporary power and energy systems, supporting essential operational and planning functions including short‐term load prediction, electricity price forecasting and so on. Transformer‐based models achieve state‐of‐the‐art results by embedding series into tokens, but their quadratic time and space complexity limits the practical applications in real‐world scenarios. In this paper, we propose to chunk the tokens before feeding them to the encoder, in which attention calculations between tokens are limited to within the same chunk. Leveraging the frequency domain characteristics of the series, we employ locality sensitive hashing (LSH) techniques to ensure that series with similar frequency patterns are assigned to the same chunk. By fixing the chunk size, our method reduces the complexity of the transformer‐based model from to . Within each chunk, distilled features are concatenated to incorporate global information. Additionally, the running score update mechanism allows our framework to adapt the chunking strategy according to the data distribution. Extensive experiments demonstrate that, across multiple real‐world datasets and models, the proposed method maintains performance comparable to the original models while substantially reducing computational complexity and memory usage.