An Enhanced Machine Learning Framework for Network Anomaly Detection

Given the increasing volume and sophistication of cyber-attacks, there has always been a need for improved and adaptive real-time intrusion detection systems. Machine learning algorithms have presented a promising approach for enhancing their capabilities. This research has focused on investigating the impact of different dimensionality reduction approaches on performance, and we have chosen to work with both Batch PCA and Incremental PCA alongside Logistic Regression, SVM, and Decision Tree classifiers. We started this work by applying machine learning algorithms directly on pre-processed data, then applied the same algorithms on the reduced data. Our results have yielded an accuracy of 98.61% and an F1-score of 98.64% with a prediction time of only 0.09 s using Incremental PCA with Decision Tree. We also have obtained an accuracy of 98.44% and an F1-score of 98.47% with a prediction time of 0.04 s from Batch PCA with SVM, and an accuracy of 98.47% and an F1-score of 98.51% with a prediction time of 0.05 s from Incremental PCA with Logistic Regression. The findings demonstrate that Incremental PCA offers near real-time IDS deployment in large networks.