Intrusion Detection System-Based Network Behavior Analysis: A Systemic Literature Review

An Intrusion Detection System (IDS) in cyberspace, as of now, plays primarily as a means of detecting illegal access and activity in a network. Due to the rapidly evolving cyber threats, the traditional signature-based IDS have started losing their effectiveness, leading to the emergence of advanced alternatives to these traditional technologies, such as Network Behavior Analysis (NBA). Unlike conventional signature-based systems, NBA monitors behavioral patterns for deviations and potential threats, which is a far more flexible and powerful way of detecting intrusion. While NBA-based IDS is a growing field of interest, the existing research in this area is mostly disoriented, mostly concentrating on single features like machine learning, deep learning algorithms, specific detection processes, or unique environments such as IoT and cloud systems. This systematic literature review (SLR) follows the guidelines proposed by Kitchenham to collect various studies, highlights research gaps, and provides an overview of the existing evidence. Spanning literature from January 2014 to April 2024, it comprehensively highlights the methods, datasets, types of detectable cyber-attacks, performance metrics, and the challenges that besiege existing NBA-based IDS. This shows the urgency for much more flexible and robust solutions, i.e., providing solutions through advanced Artificial Intelligence (AI) techniques in response to the increasing cyberspace complexities. Therefore, this review provides fundamental perspectives for researchers and practitioners and makes an important contribution towards stimulating future research efforts to design more effective and robust IDS solutions.