Optimizing Machine Learning-based Intrusion Detection Systems: A Comparative Study on KDD99 and BKIDS2025 Datasets

The increasing sophistication and frequency of cyberattacks make Intrusion Detection Systems (IDSs) essential, yet many machine learning-based IDSs studies still rely on legacy benchmarks such as KDD99, raising concerns about real-world generalization. This study optimizes and comparatively evaluates supervised IDS models on KDD99 and BKIDS2025, a contemporary dataset collected from a production university network (1,463,847 labeled flows over 30 days) covering modern protocols and usage patterns, including TLS 1.3, DNS-over-HTTPS, and IoT traffic. Five supervised models — Logistic Regression (LR), Random Forest (RF), XGBoost (XGB), Voting, and Stacking — are assessed under a unified pipeline using precision, recall, F1-score, and false alarm rate. Ensemble methods consistently outperform individual classifiers across both datasets. The Stacking Classifier (SC) achieves the best results, reaching 99.8% recall on KDD99 and 96.0% recall with a 97.2% F1-score on BKIDS2025. A Wilcoxon signed-rank test supports the statistical significance of the ensemble advantage, and runtime profiling indicates sub-millisecond inference latency, enabling real-time deployment. The performance drop on BKIDS2025 highlights the limitations of benchmark-only evaluation and motivates routine validation on up-to-date, real-world datasets for robust IDS design.