T‐<scp>VAE</scp>: Transformer‐Based Variational <scp>AutoEncoder</scp> for Perceiving Anomalies in Multivariate Time Series Data

ABSTRACT Anomaly perception in multivariate time series data has crucial applications in various domains such as industrial control and intrusion detection. In real‐world scenarios, the sequence information in multivariate time series data, which encompasses the temporal order and dependencies among high‐dimensional samples and features, can be complex and nonlinear. Additionally, the time series data often exhibit high volatility and are interspersed with noise data. These factors make anomaly perception in multivariate time series challenging. Despite the recent development of deep learning methods, only a few are able to address all of these challenges. In this paper, we propose a Transformer‐based Variational AutoEncoder (T‐VAE) for anomaly perception in multivariate time series data. The T‐VAE consists of two sub‐networks, the Representation Network and the Memory Network, and achieves end‐to‐end jointly optimisation. The Representation Network leverages self‐attention mechanisms and residual network structures to capture sequence information and metaphorical patterns from multivariate time series data. The Memory Network employs a Variational AutoEncoder to learn the distribution of normal data. It employs Maximum Mean Discrepancy to approximate the distribution of high‐volatility and noisy data to the distribution of the normal data. We evaluate T‐VAE on five datasets, showing superior performance and validating its effectiveness and robustness through comprehensive ablation studies and sensitivity analyses.