Multi-Ship Collision Avoidance in Inland Waterways Using Actor–Critic Learning with Intrinsic and Extrinsic Rewards

Inland waterway navigation involves complex traffic conditions with frequent multi-ship encounters. Benefiting from its straightforward structure and robust adaptability, reinforcement learning has found applications in navigation. This article proposes a deep actor–critic collision avoidance model which is based on the weighted summation of intrinsic reward and extrinsic reward, overcoming the sparsity of the reward function in navigation tasks. For the proposed algorithm, the extrinsic reward considers factors of collision risk, economic reward, and penalties for violating collision avoidance rules, while the intrinsic reward explores the novelty of agent states. The optimization of the own ship’s actions is achieved through the utilization of a weighted summation of these two types of rewards, providing valuable guidance for decision-making in a symmetrical interaction framework. To validate the performance of the proposed multi-ship collision avoidance model, simulations of both two-ship encounters and complex multi-ship scenarios involving dynamic and static obstacles are conducted. The following conclusions can be drawn: (1) The proposed model could provide effective decisions for ship navigation in inland waterways, maintaining symmetrical coordination between vessels. (2) The hybrid reward mechanism successfully guides ship behavior in collision avoidance scenarios.