A comparative study of machine learning and deep learning models in binary and multiclass classification for intrusion detection systems

Network infrastructure evolution has significantly expanded the attack surface, leading to increasingly complex and sophisticated cybersecurity threats. Traditional rule-based intrusion detection systems (IDS) often fail to detect emerging attack vectors, prompting the need for intelligent, data-driven approaches. This study evaluates and compares the performance of machine learning (ML) and deep learning (DL) models for network intrusion detection. Two publicly available datasets were utilized: a binary-labeled software-defined networking (SDN) dataset and a multiclass industrial control system dataset based on the IEC 60870-5-104 protocol. Preprocessing steps included normalization, label encoding, and a 70:10:20 train-validation-test split. Seven models, Random Forest, Decision Tree, K-Nearest Neighbors, XGBoost, Convolutional Neural Network, Gated Recurrent Unit, and Long Short-Term Memory, were trained and evaluated using precision, recall, and F1-score. The Random Forest model achieved the highest F1-score of 93.57% on the IEC 60870-5-104 dataset, while XGBoost attained a near-perfect F1-score of 99.97% on the SDN dataset. These results outperform comparable models in the literature and offer practical insights for selecting effective IDS solutions based on classification type and dataset structure. • Evaluate classical and deep models for both binary and multiclass tasks. • Compare best model’s results to top models reported in prior literature. • Analyze performance, training time, and CPU/GPU use, often ignored in studies. • Discuss class imbalance, bias, and feature size for real-world deployment. • Show RF and XGBoost often beat deep models despite being simpler.