Machine Learning for Intrusion Detection and Prevention Systems: A Review of Approaches and Challenges

Purpose – Machine learning (ML) and deep learning (DL) have emerged as the dominant paradigm for intrusion detection and prevention systems (IDPS), and detection accuracies reported for benchmark datasets such as CIC-IDS2017, UNSW-NB15, and NSL-KDD now consistently surpass 99%. To situate this progress, the present review examines the main families of ML-based IDPS mechanisms, encompassing classical supervised learning, deep and hybrid architectures, unsupervised and semi-supervised anomaly detection, and ensemble and federated approaches. Design/methodology/approach – Representative and highly cited studies were found in the major scholarly databases and arranged according to detection strategy, deployment setting, and learning paradigm. The recurring challenges that restrict their operational deployment were consolidated, namely class imbalance, weak cross-dataset generalization, adversarial fragility, limited interpretability, and computational cost. Findings – The synthesis exposes a structural bias in the literature, in that the vast majority of studies assess detection quality on full, post-hoc flow records under the implicit assumption of unbounded inspection capacity. In contrast, there is relatively little research on early flow classification, which involves making decisions based on the initial packets of a flow before it concludes. This capability is crucial for inline prevention, reducing latency, and optimizing resource-aware scheduling. Originality – The review shows that the disparity between offline accuracy and online, early, and resource-limited decision-making constitutes the most consequential open problem for practical IDPS, and it outlines a corresponding research agenda that reframes detection as an early, resource-aware decision problem rather than a purely accuracy-driven one.