Client: Cross-variable linear integrated enhanced transformer for multivariate long-term time series forecasting

Long-term time series forecasting (LTSF) is crucial in modern society, playing a pivotal role in facilitating long-term planning and developing early warning systems. While many Transformer-based models have recently been introduced for LTSF, a doubt has been raised regarding the effectiveness of attention modules in capturing cross-time dependencies. In this study, we design a mask-series experiment to validate this assumption and subsequently propose the ”Cross-variable Linear Integrated ENhanced Transformer for Multivariate Long-Term Time Series Forecasting” ( Client ), an advanced model that outperforms both traditional Transformer-based models and linear models. Client employs linear modules to learn trend information and the enhanced Transformer module to capture cross-variable dependencies. Meanwhile, the cross-variable Transformer module in Client simplifies the embedding and position encoding layers and replaces the decoder module with a projection layer. Extensive experiments with nine real-world datasets have confirmed the SOTA performance of Client with the least computation time and memory consumption compared with the previous Transformer-based models. Our code is available at https://github.com/daxin007/Client . • Identified the limitations in conventional Transformer models using a comprehensive mask-series experiment and proposed a novel method to address this issue. • The proposed Client model combines a linear model to capture linear trends with a cross-variable Transformer model to capture non-linear information and dependencies between variables. It emphasizes the importance of cross-variable attention over cross-time attention for LTSF tasks. • Client achieves state-of-the-art performance on nine real-world datasets, while significantly reducing training time and memory consumption compared to both conventional Transformer models and previous SOTA models, such as TimesNet and PatchTST.