A Fast-Convergence, Induced Dynamic Spectrum Access Based on Accelerated Q-Learning for Cognitive Radio Networks

In New Radio Unlicensed (NR-U) networks, spectrum resources are shared to ensure quality of service. However, the proliferation of wireless devices and the near-full allocation of sub-6 GHz frequency bands have imposed significant challenges on spectrum access conflicts. Reinforcement learning (RL) offers a promising solution to address the dynamic spectrum access (DSA) issues for cognitive users in heterogeneous spectrum environments. However, existing RL algorithms typically require a large number of iterations to achieve convergence, causing spectrum resource wastage before being converged. To mitigate this issue, this paper proposes a fast-convergence, induced dynamic spectrum access method based on an accelerated Q-Learning model and Long Short-Term Memory (LSTM) networks. This approach aims to ensure convergence while simultaneously reduce the number of required iterations. Firstly, an online learning model, built on LSTM, is employed to predict the probability of spectrum state by exploiting the temporal patterns inherent in spectrum usage. Then, the prediction outcomes are subsequently used as induction factors for the spectrum access process. Finally, the induction factors are integrated with a Q-Learning model to construct a novel joint Q-value strategy matrix. After fewer iterations of spectrum access, an optimal access strategy is obtained. Simulation results demonstrate that the proposed method achieves faster convergence rate than existing Q-Learning and Deep Q-Network (DQN) algorithms, and reduces the number of iterations by 50% and 80%, respectively. Additionally, compared to the general Q-Learning algorithm, the proposed method demonstrates higher stability in terms of available network capacity.