Novel Deep Reinforcement Learning‐Based Optimized Ensemble Approaches for <scp>IoT</scp> Network Intrusion Detection

ABSTRACT The rise in network intrusions has led to significant consequences, including privacy violations, financial losses, and unauthorized data transfers. Attackers exploit vulnerabilities in network systems, compromising security and disrupting services. Traditional intrusion detection systems (IDS) often face challenges such as false positives, delayed threat identification, and poor detection of minority attack classes. To address these issues, this research proposes advanced deep reinforcement learning with deep learning‐based NIDS for improved threat detection and mitigation. Data from IoT‐2023, BoT‐IoT, CIC‐IoT 2023, and RT‐IoT 2022 datasets are preprocessed through null value handling, data cleaning, one‐hot encoding, and Min‐Max normalization. To enhance the detection of minority attacks, the Tabular Auxiliary Classifier Generative Adversarial Network (TACGAN) is employed for synthetic data augmentation. Feature extraction is performed using the Graph Sample and Aggregate Attention Network (GSAAN), which captures basic, content, and traffic‐based features. Significant features are selected using the Mountaineering Team‐Based Optimization (MTBO). Attack classification is carried out using a novel ensemble of the Improved Double Deep Q‐Network (IDDQN) and Deep Autoregression Feature Augmented Bidirectional LSTM (DAF‐BiLSTM), which is termed the OptIDQDBiLSTM approach, ensuring robust learning of spatial and temporal dependencies. Hyperparameter tuning is optimized using the Boosted Wild Horse Optimization Algorithm (BWHOA). Experimental results show that the proposed approach outperforms existing IDS methods, achieving higher detection rates, improved accuracy, and a reduced false alarm rate while maintaining computational efficiency. While comparing with existing state of the art approaches, the proposed approach surpasses existing methods with over 99.64% accuracy, 99.34% precision, and 99.42% recall. These findings demonstrate the effectiveness of deep reinforcement learning in enhancing network security against evolving cyber threats.