Multi-ship Autonomous Collision Avoidance Decision-making Algorithm Based on MER-D3QN

To meet the higher requirements of Maritime Autonomous Surface Ships (MASS) development for autonomous collision avoidance technologies, this paper proposes a multi-ship autonomous collision avoidance decision-making algorithm based on MER-D3QN (Dueling Double Deep Q-Network with mixed experience replay). Firstly, a Reinforcement Learning (RL) test system based on the Markov Decision Process (MDP) with the heterogeneous multi-agent system (HMS) test environment is established to determine the distributed learning-execution mechanism. Secondly, a reward function is designed by considering various factors that affect ship collision avoidance from a global perspective. Meanwhile, the D3QN network is used to train the algorithm model with mixed experience replay technology, which can accelerate the model's convergence speed. Finally, the algorithm's performance is validated using the classic Imazu problem as the testing set. The experimental results show that this algorithm enables all ships to complete collision avoidance tasks safely and efficiently in compliance with the International Regulations for Preventing Collisions at Sea (COLREGs) requirements. Furthermore, the trained model can be applied in the HMS with any number of agents, providing a new research idea for the field of ship autonomous navigation.