Toward smooth and efficient autonomous berthing: A PPO-E2E stabilization control approach

To address poor navigation smoothness and maneuvering continuity in autonomous berthing, this paper proposes a berthing stabilization control method using the Proximal Policy Optimization-based End-to-End (PPO-E2E) algorithm. First, a ship dynamics model incorporating hydrodynamic effects and environmental disturbances is developed to enhance simulation fidelity. The berthing stabilization problem is then formulated as a Markov decision process and solved using PPO-E2E. A composite reward function is designed to jointly improve berthing accuracy, motion smoothness, and maneuvering continuity. It combines an integral heading-variation penalty, which suppresses unnecessary heading adjustments, with a distance-based reward that guides the vessel toward the target berth. This reward structure promotes stable and continuous control behavior during policy optimization. Simulations conducted under four representative berthing scenarios show that PPO-E2E reduces cumulative heading variation by approximately 32.3% relative to Trust Region Policy Optimization (TRPO) and 19.6% relative to a baseline PPO method, thereby achieving smoother trajectories with fewer unnecessary heading adjustments. Full-scale ship experiments further confirm the simulation findings, yielding an average berthing position error of 0.82 m and an average heading error of 0.155 rad. These findings demonstrate the feasibility of the proposed method for practical engineering applications.