Reevaluating the Potential of a Vanilla Transformer Encoder for Unsupervised Time Series Anomaly Detection in Sensor Applications

Sensors generate extensive time series data across various domains, and effective methods for detecting anomalies in such data are still in high demand. Unsupervised time series anomaly detection provides practical approaches to addressing the challenges of collecting anomalous data. For effective anomaly detection, a range of deep-learning-based models have been explored to handle temporal patterns inherent in time series data. In particular, Transformer encoders have gained significant attention due to their ability to efficiently capture temporal dependencies. Various studies have attempted the architectural improvements of Transformer encoders to address the inherent complexity of time series data analysis. Unlike the previous studies, this work demonstrates that a vanilla Transformer encoder-based framework remains yet a competitive model for time series anomaly detection. Instead of architectural modification of the Transformer encoder, we identify key design choices and propose an asymmetric autoencoder-based framework incorporating those design choices with a vanilla Transformer encoder and a linear layer decoder. The proposed framework has been evaluated on a range of unsupervised time series anomaly detection benchmarks, and the experimental results show that it achieves performance that is either superior or competitive compared to state-of-the-art models.