GSTGPT: A GPT-Based Framework for Multi-Source Data Anomaly Detection

Anomaly detection is a critical approach for ensuring the security of microservice systems. In recent years, deep sequence models have been widely applied to transform sequence modeling into a language modeling problem. However, the objective of training sequence models with language modeling loss is not directly aligned with anomaly detection. Moreover, the diverse data types in microservice systems—namely metrics, logs, and traces—exhibit asynchrony and complex interdependencies. Existing methods based on deep sequence models, such as LogBERT and TranAD, can only account for a limited number of data modalities, failing to fully utilize multi-source data and effectively handle the interrelationships among multiple modalities. To address this, we propose a multimodal anomaly detection framework based on a generative pre-trained language model (GPT), named GSTGPT. GSTGPT represents multi-source data as a feature graph, with metrics and logs as node features and traces as edge features. Additionally, we model feature interactions and dependencies within sequences using spatio-temporal attention and enhance the model’s focus on critical features through feature augmentation. Experimental results on two real-world datasets demonstrate that GSTGPT achieves an F1 score of 0.967, an 8.3% improvement over baseline methods, significantly outperforming them.