ALogSCAN: A Self-Supervised Dual Network for Adaptive and Timely Log Anomaly Detection in Clouds

Logs are prevalent in modern cloud systems and serve as a valuable source of information for system maintenance. Over the years, many supervised, semi-supervised, and unsupervised log analysis methods have been proposed to detect system anomalies. In particular, semi-supervised methods have garnered increasing attention as they balance reduced labeled data requirements and optimal detection performance, contrasting with their supervised and unsupervised counterparts. However, existing semi-supervised log analysis methods often suffer from practical challenges, such as log instability, imbalanced class data, and labeling dependency, which are pervasive issues in real-world systems. To address these challenges, we propose ALogSCAN, a self-supervised method to detect anomalies at the host level of cloud systems. ALogSCAN introduces the Dynamic Frequency-based Log Filtering (DFLF) technique to mitigate the potential bias introduced by highly frequent log messages, thereby focusing more on infrequent yet critical log messages. Moreover, the self-supervised nature of ALogSCAN eliminates the need for time-consuming manual labeling of log data, and enables the DFLF technique to continuously adapt to evolving log sequences, maintaining robustness against unstable log data. We have evaluated ALogSCAN on two widely used public datasets and one private dataset from Ericsson Research, and the experimental results demonstrate its effectiveness, consistently outperforming existing methods in various scenarios.