Synergizing Feature Selection and Engineering for Enhanced Machine Learning and Deep Learning in Network Intrusion Detection Systems

With the rise of sophisticated cyber threats, Network Intrusion Detection Systems (NIDS) have become critical in securing digital infrastructures. This paper addresses the limitations of traditional NIDS by applying advanced machine learning (ML) and deep learning (DL) techniques to the NSL-KDD dataset. Feature selection methods such as Student’s T-Test and Kruskal-Wallis Test were combined with SMOTE to enhance model performance. Three ML models—Support Vector Machine (SVM), Random Forest (RF), and Gradient Boosting (GB)—and three DL models—Dense Neural Networks (DNN), Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM)—were evaluated. Among the models, Random Forest achieved 99.91% accuracy and 1.00 precision, while LSTM delivered the highest ROC-AUC score of 0.999947. These results demonstrate the effectiveness of integrating feature selection and DL techniques, significantly improving NIDS performance in detecting complex cyber threats. The findings contribute to the development of scalable and robust intrusion detection systems for modern network security.