OST: Offline Spatial-Temporal Actor-Critic Learning for Robot Crowd Navigation

In densely crowded environments, robot crowd navigation tasks are typically approached using online reinforcement learning algorithms. However, the complex and dynamic nature of pedestrian movements significantly reduces the sample efficiency of online reinforcement learning. To address this issue, this paper proposes an offline spatial-temporal actor-critic learning method for robot crowd navigation. Specifically, a novel SARSA-like loss function is defined to update the separate value function, eliminating the need for behavior regularization or constraints. Furthermore, a spatial-temporal transformer is integrated into offline actor-critic learning to capture spatial-temporal features from offline pedestrian-robot interactions, thereby optimizing the sampling efficiency of the navigation algorithm. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art methods in both qualitative and quantitative analyses.