Optimizing Machine Learning-Based Network Intrusion Detection System with Oversampling, Feature Selection and Extraction

Network security is a global challenge that requires intelligent and efficient solutions. Machine Learning (ML)-based Network Intrusion Detection Systems (NIDS) have been proven to enhance accuracy in detecting cyberattacks. However, the main challenges in implementing ML-based IDS are dataset imbalance and large dataset size. This research addresses these challenges by applying oversampling techniques to balance the dataset, feature selection using random forest to identify the most relevant features, and feature extraction using Principal Component Analysis (PCA) to further reduce the selected important features. Additionally, K-fold cross-validation is used to test the features to minimize bias and ensure the model does not suffer from overfitting, while Optuna is implemented to automatically optimize model parameters for maximum accuracy. Since IDS performance deteriorates with high-dimensional features, the combination of methods used is evaluated based on feature selection applied to the model using datasets wtih 45 features selected from UNSW-NB15, 78 features from CIC-IDS-2017, and 80 features from CIC-IDS-2018 using various ML algorithms. The results demonstrate that the combination technique with feature selection, along with maximum optimization for each model significantly improves performance on large and imbalanced datasets reaching 99% accuracy compared to conventional methods in network traffic analysis.