Comprehensive Comparison of Streaming Clustering-Based IDS: Evaluation Across Clustering Families with a Novel Ensemble Classifier

Clustering is an effective method for detecting network intrusions, especially due to its unsupervised nature and fast reactivity, which is particularly advantageous for the identification of zero-day attacks. Despite its potential, Intrusion Detection Systems (IDS) often uses clustering algorithms based on their simplicity, without rigorous comparative analysis, and ignores important limitations of some algorithms such as preset cluster numbers and inflexible cluster shapes. These shortcomings hamper the effective use of network traffic data to detect intrusion, especially given its dynamic nature. In this article, we evaluate the effectiveness of various clustering algorithms—specifically partitioning, hierarchical, and density-based methods—in detecting anomalies in network traffic. We focus on analyzing the formation and characteristics of anomaly clusters within these approaches and propose a complete evaluation methodology suitable for intrusion detection using streaming data to be closer to real-world scenarios. Our results reveal that no single clustering algorithm universally excels on every dataset, prompting us to introduce an ensemble clustering approach that brings together the strengths of different classes of algorithms. Preliminary tests demonstrate the superior performance of our approach on three IDS datasets with a good accuracy and F1-score greater than 0.8 for all of them.