Realtime Spectrum Monitoring via Reinforcement Learning – A Comparison Between Q-Learning and Heuristic Methods

Due to technological advances in the field of radio technology and its availability, the number of interference signals in the radio spectrum is continuously increasing. In order to enforce standards and keep emergency frequencies open, regulatory teams are usually active, which must be able to detect interference signals in a timely fashion. For this purpose, specialized receivers are deployed to perform spectrum monitoring. This paper compares the performances of two different approaches regarding resource management: (i) linear frequency tuning as a heuristic approach and (ii) a Q-learning algorithm from the field of reinforcement learning. To test the methods under investigation, a simplified scenario was designed comprising two receiver channels monitoring ten frequency bands with nonuniform signal activity. For this setting, it is shown that the Qlearning algorithm has a significantly higher detection rate than the heuristic approach at the expense of a smaller exploration rate.