Sequential Reinforcement Learning-Based Waveform Adaptation for Dynamic Decision-Making in Tracking

This paper considers a waveform design problem for tracking a target in an unknown and time-varying environment. The varying conditions are due to potentially high interference levels in different frequency sub-bands as the radar is sharing spectrum with a wireless communication system. We propose an approach that sequentially improves tracking conditions using <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$Q$</tex> learning. This is a reinforcement learning method that does not require prior knowledge or training under dynamically-varying conditions. In our formulation, both the transmit waveform and the transmission frequency sub-band are selected at each time step based on suitably designed cost functions appropriately designed costs. We integrate the Q-learning algorithm with a sequential Monte Carlo filtering method, and we demonstrate the improved tracking performance using simulations.