Out of Distribution Generalization: KPI vs Spectrogram Based Jamming Classification in 5G

Nowadays, 5G network deployments and use cases increasingly rely on Artificial Intelligence to enhance network security. Machine Learning models can be leveraged to detect and classify attacks on the network-such as jamming-using Key Performance Indicators (KPIs) like SINR, MCS, and CQI as training data. Another prevalent method involves using spectrograms as training data, effectively transforming the task into an image classification challenge. In dynamic environments like 5G, encompassing every possible scenario when collecting training data is impractical. Hence, it’s crucial to develop resilient models that maintain high accuracy when exposed to data of a different distribution in the real world-a concept known as Out-of-Distribution (OOD) Generalization. This study compares the accuracy and resilience of models created using these two methodologies. Initially, models were evaluated using their respective hold-out datasets, and later with OOD test data incorporating first attenuated 5 G signals data and then diverse traffic types data. Our results show that spectrogram-based models achieved 100% accuracy on respective hold-out datasets, lowest of 96% on different traffic type OOD test data, and lowest of 66% on attenuated OOD test data. In contrast, the best-performing KPI-based model reached a maximum of 92% on its respective hold-out dataset but showed poor performance on OOD test sets, at times misclassifying everything. We thus infer that spectrogram-based models exhibit superior accuracy and resilience in 5G jamming detection and classification.