A feature selection-driven machine learning framework for anomaly-based intrusion detection systems

Abstract In light of rapid technological developments, a marked rise in global internet usage has contributed to increased sensitive data flow across networks. This increase leads to the diversification of malicious attacks and makes cyber security requirements more evident. In order to ensure network security, intrusion detection systems stand out as an essential component. Intrusion detection systems detect suspicious and malicious activities over network traffic, allowing network administrators and experts to monitor current threats continuously. In anomaly-based systems, machine learning approaches are applied to identify abnormal attempts in network traffic. This study presents a feature selection framework for anomaly-based attack detection systems by combining machine learning and heuristic algorithms. This proposed study aims to improve the performance of IDSs in terms of both time and attack detection by selecting features with heuristic approaches. In the proposed approach, PSO, FPA, DE feature selection methods and LR, DT, RF, KNN, NB, GB, LDA, QDA, AdaBoost, and NN machine learning algorithms are used to perform anomaly-based comparative analyses on KDDCup99, NSL-KDD, UNSW-NB15, CSE-CIS-IDS2018 datasets. Analyses conducted on these datasets with various features demonstrated that models employing feature selection achieved an approximate two-hundred-percent improvement in time efficiency compared to models that did not utilize feature selection. It has been determined that DE, PSO, and FPA, which are used for feature selection, provide high-accuracy outputs when combined with different classifiers. When the analysis results are assessed according to the specified criteria, the highest F1-Score values achieved are as follows: 0.9972 for the DE method in GB, 0.9969 for the PSO method in GB, and 0.9948 for the FPA method in GB, on the KDD CUP 99 dataset. In the NSL-KDD, used as the second dataset, the DE method achieved a score of 0.9713 in GB, the PSO method reached 0.9112 in DT, and the FPA method obtained 0.9894 in RF, respectively. In the third dataset, UNSW-NB15, the DE method achieved a score of 0.9507 in DT, the PSO method reached 0.9068 in DT, and the FPA method obtained 0.8924 in NN. Finally, in the CSE-CIC-IDS2018 dataset, the highest scores achieved using the RF algorithm were 0.99986 for the DE method, 0.99989 for the PSO method, and 0.99987 for the FPA method, based on feature selection. The obtained results underscore the critical role of dataset generation processes and network traffic dynamics in enhancing the performance of intrusion detection systems. Additionally, the significance of feature selection was highlighted. These findings offer valuable insights and present opportunities for further advancements in future research.