Human-Navigable Ship-Handling Support Using Improved Deep Deterministic Policy Gradient for Survey Line Tracking

This study presents a human-navigable ship-handling support system that employs artificial intelligence (AI) for survey line tracking. AI was developed using the Deep Deterministic Policy Gradient (DDPG), a type of deep reinforcement learning (DRL), and was evaluated through experiments conducted with a research vessel. The experiments revealed several issues inherent to DRL that required improvement. The first issue was the asymmetry observed in the policy learned through the DDPG. To address this, a learning approach that utilizes symmetric training data and symmetry-constrained actor and critic neural networks was proposed. The second issue was excessive steering during tracking maneuvers. To mitigate this, an objective function for actor learning that incorporates a cost term to suppress the magnitude of actions was proposed. The third issue was the frequent oscillation of actions. To resolve this, improved conditioning for action policy smoothing was introduced in the objective function to smooth actions appropriate to the situation. A subsequent experiment at sea was conducted to evaluate the improved AI-based ship-handling support system. As a result, precise path tracking performance with minimal operator discomfort and smooth control actions was achieved through manual ship handling guided by AI-generated instructions under actual sea conditions.