Unmanned Surface Vehicle Navigation Under Disturbances: World Model Enhanced Reinforcement Learning

Rapidly developing deep reinforcement learning (DRL) methods are suited to address the increasingly complex task requirements of unmanned surface vehicles (USV) that traditional methods cannot resolve. However, current DRL methods fail to make full use of the USV dynamics characteristics and do not consider unknown external disturbances. This article presents a state prediction reinforcement learning (SPRL) framework, to enhance the robustness against disturbances in obstacle avoidance navigation tasks for USVs. A novel latent robust world model (LRWM) with double latent features is elaborated based on probabilistic graphical model (PGM) to learn the transition of the USV navigation and predict future latent features against disturbances. A motion feature is learned to capture the impact of environmental disturbances on USV dynamics, while a prediction feature is extracted to predict future motion states. The proposed SPRL method can provide both the actor and the critic with essential prior information, thereby facilitating its application to actor–critic-based DRL methods. We performed field experiments in a lagoon environment, demonstrating the proposed method's deployability on practical USV platforms and its effectiveness in obstacle avoidance navigation under unknown disturbances.