HSTRL: A Hybrid Spatiotemporal State Model for Mobile Agent Autonomous Navigation

The field of intelligent robotic control has long been engaged in the study of autonomous navigation for mobile agent in complex dynamic environments. In order to address this significant challenge, a hybrid spatiotemporal enhanced reinforcement learning method (HSTRL) is proposed. The novel HSTRL model is designed to comprehensively extract and represent the spatiotemporal state information generated during agent-environment interactions. The model incorporates a network designed to extract both spatial and temporal states, thereby enabling an enhanced representation of hybrid spatiotemporal states, which is of paramount importance for the handling of the continuous changes inherent in complex dynamic environments. The HSTRL model employs value-based deep reinforcement learning to identify the optimal navigation strategy, thereby enabling agent to navigate safely and efficiently in such environments. The advantages of our method include the capacity to generate safe and efficient navigation strategies by leveraging comprehensive spatiotemporal information. A series of experiments have demonstrated the superiority of the proposed HSTRL model in agent navigation tasks, outperforming previous methods in both success rate and efficiency.