MARL to Choose Actions On-the-Fly in a Cognitive Radar System

Task scheduling for radar systems have generally taken a block approach, scheduling a given set of tasks, with predetermined task parameters, to optimize some cost function. This paper takes a contrasting, on-the-fly, approach, determining what the radar(s) must “do next” to maximize their understanding of their operating environment. Specifically, we develop reinforcement learning (RL) for joint tracking and surveillance in a multi-function radar system. We further develop a multi-agent reinforcement learning (MARL) system to coordinate the actions of two radars. In this paper, we define actions as the act of viewing an azimuth around a radar. We outline the system's architecture, environmental dynamics, RL agents employed, and comprehensive metrics for evaluating agents' performance. Our approach presents significant challenges due to the ambiguity in defining a state space, the extensive time required to train agents, and the large number of tunable RL hyperparameters. Additionally, the fundamentally opposed nature of tracking and searching tasks introduces a dichotomy in the requirements of a reward function for the RL agent. The findings of this paper provide insight and advancements in the field of cognitive radar systems.