Advancing intrusion detection with ML and deep learning: a comparative approach

In network security, Intrusion Detection Systems (IDS) is increasingly important due to rise in cyber-attacks. To ensure accurate detection and prevention, IDS leverages Machine Learning (ML) and Deep Learning (DL) algorithms on benchmark datasets; CICIDS2017 and NSL-KDD. These datasets, with diverse attack types, are ideal for training and testing. Traditional IDS struggle to detect zero-day attacks due to lack of predefined signatures, leading to higher false positive rates and vulnerabilities. Our proposed methodology focuses on a DL-based NIDS. We pre-process and normalize data from CICIDS2017, NSL-KDD, and a custom home network dataset. Supervised ML models, including Random Forest, Decision Tree, and Gaussian Naive Bayes, were applied to CICIDS 2017, achieving high accuracy. For DL, BiLSTM and Transformer models were trained on NSL-KDD for 30–60 epochs, with accuracy rates of 87%–99%. We utilized dual approach of normalization technique using Log Transformation and enhanced the Log Transformation by adding weightage to the features and retrained the model LSTM-CNN on NSL-KDD. Additionally, an LSTM-CNN model was applied to the Home Network dataset, the accuracy result of H-N Version1 was 100%.