Adaptive network anomaly detection using machine learning approaches

Abstract This research aims to develop a Network Detection System (NDS) utilizing various machine learning techniques to enhance network security through anomaly detection. It evaluates the effectiveness of K-nearest neighbors (KNN), gradient boosting, support vector machines (SVM), random forests, and logistic regression in identifying deviations from normal network behavior. Furthermore, ensemble learning methods, including voting and stacking techniques, are explored to improve detection accuracy. The study proposes and tests a hybrid multi-layered stacking model using the CICIDS 2017 dataset, which encompasses both historical and modern attack patterns, providing a comprehensive benchmark for evaluation. Model performance is assessed using metrics such as accuracy, precision, recall, and F1 score. Special emphasis is placed on feature importance and reduction in dimensionality to enhance model efficiency. Additionally, the study addresses the critical challenge of minimizing false positives and false negatives for practical deployment. Results indicate that the hybrid ensemble stacking model achieves superior performance, with an accuracy of 98.79%, significantly improving network anomaly detection. The research highlights the potential for further advances through deep learning and real-time detection methodologies to improve network security in the future.