Development of a Reinforcement Learning-Based Ship Voyage Planning Optimization Method Applying Machine Learning-Based Berth Dwell-Time Prediction as a Time Constraint

Global container shipping faces increasing pressure to reduce fuel consumption and greenhouse gas (GHG) emissions while still meeting strict port schedules under highly uncertain terminal operations and met-ocean conditions. However, most existing voyage-planning approaches either ignore real port operation variability or treat fuel optimization and just-in-time (JIT) arrival as separate problems, limiting their applicability in actual operations. This study presents a data-driven just-in-time voyage optimization framework that integrates port-side uncertainty and marine environmental dynamics into the routing process. A dwell-time prediction model based on Gradient Boosting was developed using port throughput and meteorological–oceanographic variables, achieving a validation accuracy of R2 = 0.84 and providing a data-driven required time of arrival (RTA) estimate. A Transformer encoder model was constructed to forecast fuel consumption from multivariate navigation and environmental data, and the model achieved a segment-level predictive performance with an R2 value of approximately 0.99. These predictive modules were embedded into a Deep Q-Network (DQN) routing model capable of optimizing headings and speed profiles under spatially varying ocean conditions. Experiments were conducted on three container-carrier routes in which the historical AIS trajectories served as operational benchmark routes. Compared with these AIS-based baselines, the optimized routes reduced fuel consumption and CO2 emissions by approximately 26% to 69%, while driving the JIT arrival deviation close to zero. The proposed framework provides a unified approach that links port operations, fuel dynamics, and ocean-aware route planning, offering practical benefits for smart and autonomous ship navigation.