Learning to Optimize State Estimation in Multi-Agent Reinforcement Learning-Based Collaborative Detection

In this paper, we study the collaborative detection problem in a multi-agent environment. By exploiting onboard range-bearing sensors, mobile agents make sequential control decisions such as moving directions to gather information of movable targets. To estimate target states, i.e., target location and velocity, the classic works such as Kalman Filter (KF) and Extended Kalman Filter (EKF) impractically assume that the underlying state space model is fully known, and some recent learning-based works, i.e., KalmanNet, estimate target states alone but without estimation uncertainty, and cannot make robust control decision. To tackle such issues, we first propose a neural network-based state estimator, namely T<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</u>o-phase K<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AL</u>ma<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</u> Filter with <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</u>ncertainty quan<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</u>ification (WALNUT), to explicitly give both target states and estimation uncertainty. The developed multi-agent reinforcement learning (MARL) model then takes the learned target states and uncertainty as input and makes robust actions to track movable targets. Our extensive experiments demonstrate that our work outperforms the state-of-the-art by higher tracking ability and lower localization error.