VCRLog: Variable Contents Relationship Perception for Log-based Anomaly Detection

Log-based anomaly detection is crucial for software reliability assurance. System logs are semi-structured data containing constant and variable contents, both of which can provide valuable features for anomaly detection. Due to variables being heterogeneous and discrete, there is a lack of effective approaches that can comprehensively incorporate features of variables into log-based anomaly detection. In this paper, we propose VCRLog, an anomaly detection method that mines the relationships among the heterogeneous and discrete variables and extracts important features contributing to anomaly detection. Firstly, considering parsing methods cannot accurately extract variables from logs, we propose a variable extraction method based on domain knowledge. Secondly, to capture and extract the relationship feature among heterogeneous and discrete variables, we design a conceptual model based on system operation to construct variable attributed graph, which can mine important feature vectors by structural embeddings. Finally, considering constants directly express the meaning of logs, we combine relationship vectors with semantic vectors of constants to achieve transformer-based anomaly detection. Experimental results show that our proposed method can accurately detect anomalies and maintain high accuracy as the training data size decreases, outperforming existing methods. Our source code and experimental data are publicly available at https://github.com/Fridaywjy/VCRLog.