Enhanced Feature Selection with Autoencoders and Whale Optimization Algorithm for the Classification of DDoS Attacks using XGBoost based Model

Distributed Denial of Service (DDoS) attacks continue to pose a serious and evolving threat to the stability, availability, and reliability of network systems. With the rapid growth of Internet-of-Things (IoT) devices, cloud infrastructures, and software-defined networks (SDN), the scale and sophistication of DDoS attacks have also grown significantly. Traditional detection methods often struggle to cope with high-dimensional traffic data, redundant attributes, and the nonlinear interactions inherent in modern attack patterns. These limitations result in reduced detection accuracy, susceptibility to false alarms, and poor generalization across diverse attack types. To mitigate these cyber threats, this research work on an approach that integrates Autoencoders (AE), the Whale Optimization Algorithm (WOA), and Extreme Gradient Boosting (XGBoost). The AE component compresses raw traffic flows into latent feature representations, capturing nonlinear relationships and reducing noise. However, AE alone produces a broad feature space (64 features), which includes redundancy and non-informative attributes. To refine this space, WOA is employed as a metaheuristic feature selector, guided by a fitness function that balances classification accuracy with feature compactness. This process reduces the dimensionality to 40 highly discriminative features, ensuring efficiency without compromising information richness. Finally, XGBoost is applied as the classifier due to its robustness, scalability, and ability to handle both detection and multi classification of DDoS attacks. The proposed model was evaluated using CICDDoS2019, which capture diverse attack scenarios and is widely used in intrusion detection research. Baseline experiments with AE–XGBoost achieved 98.57%. By contrast, the proposed AE+WOA-XGBoost achieved 99.89% accuracy, 99.96% precision, 99.87% recall, 99.91% F1-score, and an AUC of 1.000, representing a 1.32% gain over the baseline and near-perfect classification performance. Beyond raw accuracy, the optimization process reduced computational overhead, improved generalization, and demonstrated consistent effectiveness across both datasets.