Parallel Ships Maneuvering Control Based on Deep Reinforcement Learning

Abstract This paper presents a study on the parallel maneuvering control of two ships using deep reinforcement learning (DRL). The study develops a simulation environment that models the hydrodynamic interactions between DTMB5512 and KCS when operating in close proximity. Utilizing the line-of-sight (LOS) algorithm, a two parallel ships maneuvering model (MMG) is established that considers interaction forces and wave effects, Combined with control requirements, the model frames the parallel ships maneuvering problem as a Markov decision process and designs its state and action space. The state space includes the positions and heading angles of the ships. The action space includes parameters of the Proportional Integral Derivative (PID) controller to achieve adaptive control. A DRL algorithm is introduced combining Long Short-Term Memory Networks (LSTM) with Proximal Policy Optimization (PPO). This algorithm adjusts control dynamically based on changing environmental conditions. Control laws are formulated through simulations that account for various velocities and different wave directions. The study shows that the DRL control method successfully adjusts to different wave conditions, ensuring the ships remain in position and stable. This shows the potential of DRL as a promising solution for complex maritime operations.