TiTAD: Time-Invariant Transformer for Multivariate Time Series Anomaly Detection

Anomaly detection in multivariate time series data is critical for industrial sectors such as manufacturing and aerospace. While existing methods have achieved notable success in specific scenarios, they often narrowly focus on either the temporal or spatial dimensions while overlooking their complex interdependencies. Furthermore, these approaches tend to neglect the time-invariant characteristics that are crucial for accurately capturing the spatio-temporal dynamics of the time series. To address these limitations, this paper introduces the Time-invariant Transformer for Multivariate Time Series Anomaly Detection (TiTAD), a novel framework that synergizes temporal invariance with spatio-temporal modeling. TiTAD leverages the Time-invariant Transformer, a component that excels at extracting both spatio-temporal and time-invariant features by incorporating an augmented memory mechanism. This mechanism enhances anomaly identification robustness through synergistic integration of heterogeneous feature sets. Additionally, TiTAD mitigates the Transformer’s tendency to lose temporal sequence information through the use of the Gated Recurrent Unit (GRU), thereby further enhancing the model’s capability to discern spatio-temporal patterns. The inclusion of a Feature Fusion module within TiTAD serves to refine the extracted features by adjusting their weights and minimizing redundancy, ensuring that the most relevant information is utilized for prediction and anomaly detection. Empirical evaluation on three industrial-scale benchmarks (SWaT, WADI, and SMD) demonstrates TiTAD’s superior performance compared to other methods.