Improving intrusion detection in O-RAN with synthetic data generation: A GAN and SMOTE approach

In network security, the issue of imbalanced data, where certain classes are under-represented, poses a significant challenge for intrusion detection systems (IDS). Machine learning algorithms often prioritise the majority class, resulting in poor detection of minority classes and reduced overall model accuracy. Accurately identifying these rare intrusions is essential for predicting and mitigating emerging threats. This paper presents an advanced IDS designed for Open Radio Access Networks (O-RAN) to address class imbalance by employing Synthetic Minority Over-sampling Technique (SMOTE) variants and Generative Adversarial Networks (GANs). The system was evaluated on the CICEVSE2024 dataset and a real-world O-RAN dataset. Results show that SMOTE improved the accuracy of weaker classifiers such as Naive Bayes by up to 10% (from 53.9% to 64.0%), with corresponding increases in recall and F1-score. Ensemble methods like Random Forest and XGBoost maintained high accuracy ( ≈ 81%–89%) and benefited from balanced recall when synthetic data was applied. However, GAN-generated data (CTGAN) showed little to no improvement over baseline models, and in some cases, SMOTE reduced accuracy for classifiers such as Logistic Regression and Random Forest on the O-RAN dataset (by up to 30%). These results highlight that while SMOTE variants can significantly enhance minority class detection, especially for weaker classifiers, the utility of GANs remains limited in this context. Future work should therefore focus on improving GAN-based data quality, exploring hybrid deep learning approaches, and extending IDS to real-time and multi-class scenarios.