Weighted Averaging-Based Ensemble Learning for Network Intrusion Detection

Network systems handle sensitive and confidential data, such as client information, intellectual property and proprietary assets, necessitating robust security mechanisms to ensure data confidentiality and integrity. This paper presents a comprehensive methodology that leverages Deep Learning (DL) and big data for Network Intrusion Detection (NID), aiming to identify whether assaults are occurring or not. The suggested methodology begins with data collection and pre-processing, where network traffic data is gathered, denoised and rescaled using min–max scaling. To rectify the class imbalance and improve the detection of rare attacks, the Synthetic Minority Over-Sampling Technique (SMOTE) is employed. Apache Hadoop is employed to manage large-scale data in a distributed computing environment. Feature extraction incorporates higher-order statistical features, time series analysis and traffic-based statistical features, while Enhanced Mutual Information-based Feature Selection (EMIFS) is accustomed to reduce dimensionality and optimize the DL model. The Weighted Averaging based-Ensemble Learning (WAEL) framework integrates Capsule Neural Network (CapsNet), Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) as base models. Implemented in Python, the methodology achieves superior results, with accuracy values of 98.79% on the KDD Cup 1999 dataset and 96.4% on the IDS 2018 Intrusion CSVs dataset, demonstrating WAEL’s effectiveness and reliability in NID across diverse datasets.