Tiformer:An Improved iTransformer Model for Multivariate Time Series Prediction

Due to the powerful ability of handling long-term dependencies, Transformer-based multivariate time series fore-casting (MTSF) models have shown excellent performance in recent years. During the embedding phase, such models typically process sequences point-wise across all dimensions, primarily focusing on the temporal dependencies of multiple variables at different time steps. The iTransformer model, on the contrary, embeds the complete sequence of a single variable as a whole, allowing it to better model the dependencies between different dimensions/variables, and gets an improved prediction performance. Nevertheless, we still believe that temporal dependencies across all dimensions, which are widely used in previous works, is also crucial. Therefore, this paper proposes an improved iTransformer model for MTSF, named Tiformer, in which both temporal and variable dependencies are integrated. Specifically, the FFN in iTransformer is replaced with the TFMixer module, in which a gated attention network is firstly used to capture the correlations between different time steps of a single variable, and the resulting vectors are then transposed and input into a MLP to better model the dependencies between different variables at each time step. Additionally, considering the potential non-stationarity of time series, DishTS is incorporated to predict future distribution based on historical distribution and prior knowledge, alleviating the non-stationarity issues caused by distribution shift. Experimental results on 6 real-word datasets show that Tiformer decreases MSE by an average of 6.53% and decreases MAE by an anverage of 1.56%, compared to the current best iTransformer model.