UAV-Enabled Integrated Sensing, Communication, and Control: A Constrained RL Approach

In this paper, we propose a time-division integrated sensing, communication and control (ISCC) scheme designed to dynamically enhance communication and sensing capabilities on the UAV platform. The UAV is dispatched to track a randomly moving target for capturing and transmitting sensing data to the base station via wireless communication. The goal is to leverage the ISCC framework for maximizing the cumulative sensing mutual information while guaranteeing successful data transmission by optimizing the allocation of the communication and sensing time slots together with the UAV’s control scheme. The formulated problem cannot be straightforwardly solved by off-the-shelf optimization algorithms due to the time-varying environment. To tackle this challenge, a constrained soft actor-critic (C-SAC) algorithm is developed, which dynamically switches between maximizing rewards and minimizing constraint violations to ensure robust performance in changing environments while maintaining the simplicity and efficiency of unconstrained policy optimization. Simulation results demonstrate that the proposed C-SAC algorithm outperforms dual-variable-based methods in handling the constrained problems, while extensive Monte Carlo tests confirm the robustness of the ISCC policy trained by the proposed algorithm, which adapts to varying target speeds and achieves higher cumulative mutual information compared to the point-mass UAV models.