Hierarchical Reinforcement Learning for Mapless Moving Target Navigation With Communication Interruption

Deep reinforcement learning (DRL) has shown significant potential in robot navigation, particularly for mapless moving target scenarios. Note that existing methods rely heavily on continuous communication with the target for trajectory prediction. When communication interruption (CI) occurs, the absence of real-time updates significantly reduces success rates and navigation efficiency. To bridge this gap, a novel framework is developed, namely, hierarchical reinforcement learning for CI (HRL-CI), which incorporates two key modules into the HRL architecture: adaptive sub-goal update and predicted trajectory fusion. Specifically, the adaptive sub-goal update module dynamically regulates the sub-goal update interval using the real-time comparison between the predicted and ground-truth trajectory, which constantly guarantees an opportune sub-goal adjustment for the navigation controller. The predicted trajectory fusion module exponentially weights multiple predicted trajectories to obtain the fused prediction, which reduces navigation sub-goal deviations caused by long-distance prediction errors. Comprehensive experiments demonstrate the superior performance of the HRL-CI framework, including a higher success rate, shorter navigation distance, and reduced navigation time. We’ll make our code and model weights publicly accessible at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/fzhz666/HRL-CI</uri>.