BiCAM: A Bidirectional Contextualized Attentive Model for Analyzing the Correlation of Heterogeneous Security Events

As the Internet continues to evolve, modern information technology infrastructures are constantly under attack and need to be continuously monitored for timely responses. Different devices and detection platforms generate heterogeneous security events that are sent to security operations centers, where security operators investigate those events and identify potential threats. Unfortunately, it is impossible to manually analyze such a huge number of events, leading to “alert fatigue.” Despite a substantial amount of effort having been made to aggregate redundant related alerts, the effectiveness of previous works was essentially restrained by their limited relation learning and explaining abilities. In this work, we propose the bidirectional contextualized attentive model (BiCAM), a novel contextual analysis model that uses a self-supervised deep learning approach to automatically correlate security events in relation to their bidirectional context. It is developed by designing an encoder–decoder architecture that consists of bidirectional gated recurrent units and an attention mechanism to capture both sequential and nonsequential relations of previous and subsequent alerts and provide explainability information for the security operators. In addition, we introduce a bidirectional encoder representations from transformers (BERT)-based embedding method to deal with the heterogeneity of security events, enhancing our model's accommodation to the changes of detectors. We comprehensively evaluate our model on real-world datasets containing over 11M events generated by detectors from 8 different vendors. We found that our model enables accurate, unsupervised correlation extraction; and outperforms the state-of-the-art (SOTA) work when applying event relevance to semiautomatically classify security events (e.g., the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$F1$</tex-math></inline-formula>-score of classification is improved by 4.3% and the false positive rate dropped to 1.39%).