Applications of Reinforcement Learning in Radar Spectrum Sharing with Reactive Emitters

In the last 20 years, radars have been required to share spectrum with other radio frequency (RF) systems. The substantial body of research on radar spectrum sharing has focused on collaborative approaches and non-collaborative approaches with simple interfering emitter behaviors. Modern radar systems have many degrees of freedom (DoFs) which present an opportunity to develop algorithmic techniques to harness the DoFs for effective spectrum sharing with complex interfering emitters. This paper investigates the applicability of reinforcement learning for radar spectrum sharing. To this end, we present results using Q-Learning with <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\varepsilon$</tex>-greedy action selection to inform radar waveform selection in environments containing interfering emitters with complex behaviors such as reactivity and time-varying adaptation to the changing spectrum environment. Additionally, we conduct a parametric study of the exploration rate used for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\varepsilon$</tex>-greedy action selection and assess exploration strategies for spectrum sharing problems that include emitters exhibiting both reactive and dynamic changes in behavior.