Optimizing Spectrum Access in Cognitive Radio Networks: A Reinforcement Learning Approach with Correlation-Aware Utility Maximization

The rapid proliferation of wireless communication applications across both commercial and military sectors has led to increased pressure on the limited radio spectrum, making efficient spectrum utilization a critical challenge. This study introduces an innovative methodology for enhancing frequency access in intelligent radio systems, where auxiliary users (SUs) are involved and intelligently exploit underutilized spectrum bands while minimizing interference with primary users (PUs). A reinforcement learning-based strategy is proposed, leveraging a correlation model to portray the spatial and temporal dependencies in the spectrum occupancy behavior of PUs. By modeling these correlations as a partially observable Markov decision process (POMDP), the challenge of optimizing channel access in dynamic, uncertain environments is addressed. The PERSEUS algorithm is employed to efficiently approximate the value iteration process, enabling scalable computation even with a large number of spectrum bands. The effectiveness of the introduced model is evaluated against cutting-edge approaches, including maximum a posteriori (MAP)-based state estimators and correlation-coefficient clustering algorithms, in terms of secondary network throughput and interference with PUs. Simulations demonstrate significant improvements in throughput while reducing collision rates, making this approach a promising solution for future spectrum management in cognitive radio networks.