A Trend‐Aware Transformer‐Based Approach for Improving Long‐Range Multivariate Time‐Series Forecasting With Decomposition

ABSTRACT For many years, time‐series data analysis and prediction has been extensively studied, developed, and showed great potential in dealing with multiple real‐world problems, including long‐term financial forecasting, and early extreme weather condition forecasting. The rapid evolution of deep neural networks has significantly advanced the field of time‐series forecasting, offering powerful tools for analyzing and predicting complex temporal patterns. Transformers, in particular, have gained widespread adoption for their ability to model long‐range dependencies through the multiheaded self‐attention (MHA) mechanism, making them effective for both univariate and multivariate time‐series (MTS) tasks. However, when applied to complex MTS data, most traditional transformer‐based techniques face notable challenges. The intricate temporal dependencies and cross‐variable relationships in MTS are often too complex for standard self‐attention mechanisms to fully capture. Thus, it leads to limitations in forecasting performance. Moreover, existing transformer‐based approaches frequently neglect the critical cross‐variable interactions that are essential for accurate multivariate forecasting. Moreover, previous techniques are also limited in comprehensively evaluating the temporal patterns at the series level. These temporal patterns are crucial for understanding how historical trends influence future predictions. To address these limitations, we propose a novel TAT4MTS model, which is a trend‐aware transformer‐based architecture that effectively captures intricate cross‐dependent patterns and produces enhanced series‐level representations, enabling significant improvements in long‐term forecasting accuracy. The trend‐aware projection mechanism, which is applied in our model, can assist in effectively discovering and capturing intricate cross‐dependent patterns in MTS data. As a result, it generates enhanced aggregated series‐level representations of input sequences, thereby improving its ability to model complex temporal relationships. Extensive empirical studies within real‐world multivariate weather datasets validate the effectiveness as well as the outperformance of our proposed model, comparing with previous transformer‐based forecasting baselines.