Early Detection of Malicious Activity in Log Event Sequences Using Deep Learning

The increasing frequency and severity of security incidents, particularly those involving previously unknown malicious activity, highlight the urgent need for innovative approaches to improve the cybersecurity posture of the organization. Early detection of these activities is essential for preventing security breaches and minimizing data loss. This paper deals with application of deep learning techniques for the detection of malicious activities within network log data. Our methodology involves utilizing logs obtained from prior security incidents detected by Security Information and Event Management (SIEM) to construct a model adept at identifying patterns within user-system connection sequences. In particular, benign activity logs were collected from Elastic stack during periods devoid of reported incidents to establish a baseline for normal activities. Central to our approach is the prioritization of minimizing false positives, a crucial aspect for building trust in the system among security analysts. Our model is able to detect malicious activity by analyzing only about a quarter of the sequence length, compared to detection by SIEM using manually defined detection rules. By reducing the occurrence of irrelevant alerts, our model aims to provide a high-fidelity detection system that empowers analysts to focus their attention on genuine threats and not be subjected to alert fatigue, ultimately enhancing an organization’s overall cybersecurity.