Transformers for time-series forecasting: A comprehensive survey through 2024

Time-series forecasting is directly linked to the operational optimization of social infrastructure, including electric power, transportation, weather, and infectious disease control. Various physical, statistical, and deep learning models have been developed, but all face a trade-off among interpretability, expressive power, and computational load. The Transformer, introduced in 2017, was initially hailed as a definitive solution for time-series forecasting due to its ability to capture long-term dependencies in parallel. However, a report at AAAI 2023 revealed cases where it failed to outperform even simple linear regression, casting significant doubt on its superiority. Since then, various improvements have been proposed, such as patching, channel independence, frequency decomposition, and Decoder-only foundation models. Nevertheless, as of 2025, fundamental challenges remain, including the sharpness of the loss function and the interpretability of attention. As a result, non-attention-based architectures, exemplified by GraphCast, are returning to the mainstream. This survey systematically organizes major research published from early 2023, when skepticism toward time-series Transformers grew, to the end of 2024. It formulates the basic structure of the Transformer and the task of time-series forecasting, and introduces the research progress of time-series forecasting methods using Transformers. We hope this paper will serve as a valuable reference not only for Transformer researchers but also for the broader research community engaged in the study of complex and dynamical systems.