Hybrid ensemble federated learning using SMOTE-Tomek for efficient DDoS detection on constrained edge devices over 5G networks

• A hybrid ensemble detection framework is proposed for DDoS attack classification in 5G edge networks • Combines XGBoost, Gradient Boosting, and Random Forest models using federated learning principles • Data imbalance is addressed using SMOTE and Tomek Links to enhance classification performance • Only class probability outputs are shared, reducing communication overhead and preserving privacy • Ensemble aggregation is performed using Soft Voting and Stacking with Logistic Regression as meta-learner • Evaluated on the NCSRD-DS-5GDDos dataset using realistic edge devices like Raspberry Pi 4 and 5G routers • Achieves up to 98% accuracy, precision, recall, F1-score, and 99.9% AUC-ROC with minimal latency • Demonstrates robust, scalable, and privacy-preserving DDoS detection for current and future 5G and suggested for 6G networks. The surge in connected devices within 5G edge computing ecosystems has intensified the threat of Distributed Denial-of-Service attacks, challenging the reliability and security of emerging networks. As 5G Phase 2 introduces enhanced capabilities like standalone architecture, network slicing, ultra-reliable low-latency communication, and massive machine-type communication, traditional centralized machine learning models fall short due to privacy, latency, and scalability concerns. To address this, the proposed simulation introduces a hybrid ensemble detection framework grounded in federated learning principles, where three distinct models, XGBoost, Gradient Boosting, and Random Forest are deployed on simulated clients with localized, balanced data partitions. Data imbalance is mitigated using SMOTE (Synthetic Minority Oversampling Technique) and Tomek Links (Down sampling) technique, enhancing class distribution and model performance. Instead of sharing entire models, only the class probability outputs are transmitted for aggregation, significantly reducing communication overhead while preserving privacy. For ensemble, Soft Voting and Stacking with Logistic Regression as the meta-learner are used to classify attack traffic. Evaluations on the NCSRD-DS-5GDDos dataset, sourced from devices including Raspberry Pi 4 (Waveshare 5G Hat M2), Industrial 5G Router, Wi-Fi 6 Mobile Hotspot, and Customer Premises Equipment (CPE) Box, ensuring a realistic attack simulation over various network cells. It demonstrates high detection efficacy for DDoS types like UDP Flood, SYN Flood, HTTP GET Flood, ICMP Flood, and DNS Amplification. Achieving up to 98% accuracy, precision, recall, F1-score, and 99.93% AUC-ROC with minimal latency and computational load, the method ensures robust, privacy-preserving real-time detection in 5G/6G edge networks.