A hybrid framework with uncertainty-aware cooperative fusion for robust spectrum sensing in beyond 5G cognitive radio networks

This paper presents a hybrid Gated Recurrent Unit–Long Short-Term Memory–Convolutional Neural Network (GRU–LSTM–CNN) deep learning framework designed to enhance Cooperative Spectrum Sensing (CSS) in Cognitive Radio (CR) networks. The proposed architecture combines Convolutional Neural Networks (CNNs) for spatial–spectral feature extraction, Long Short-Term Memory (LSTM) units for long-term temporal modeling, and Gated Recurrent Unit (GRU) layers for efficient short-term dependency learning. This integrated framework enables reliable detection of Primary User (PU) activity under challenging wireless environments. A reliability-aware fusion mechanism is incorporated, where each Secondary User (SU) transmits soft sensing reports containing PU-presence probability and predictive uncertainty. The key novelty is a sequential multiscale spatiotemporal representation in which CNN, LSTM, and GRU respectively learn localized spectral structure, long-term temporal dependencies, and short-term temporal variations, followed by uncertainty-aware reliability-weighted cooperative fusion. This mechanism allows the Fusion Center (FC) to assign adaptive reliability weights according to Signal-to-Noise Ratio (SNR) and confidence level. The system processes 256-QAM modulated Orthogonal Frequency Division Multiplexing (OFDM) signals transformed into time–frequency representations using the Short-Time Fourier Transform (STFT). For comparative purposes, an Orthogonal Time Space Modulation (OTSM) waveform is additionally used as a reference signaling scheme in the BER and PSD analyses (consistently denoted OTSM in Figs. 9, 10 and 11). The model is trained using a composite loss function combining Binary Cross-Entropy (BCE), Brier Score, and sparsity loss to improve detection accuracy, probability calibration, and communication efficiency. Simulation results over AWGN, Rayleigh, and Rician fading channels show that the proposed GRU–LSTM–CNN framework achieves higher Probability of Detection (Pd), lower Probability of False Alarm, improved BER performance, and reduced Power Spectral Density (PSD) compared with conventional Energy Detection (ED), Matched Filter (MF), and existing deep learning models. The proposed model achieves Pd ≈ 1 at − 4 dB SNR and maintains strong robustness under 15–25% channel estimation error, confirming its suitability for next-generation CR and IoT communication systems.