SSRL: A Clustering-Based Reinforcement Learning Approach for Efficient Ship Scheduling in Inland Waterways

Efficient ship scheduling in inland waterways is critical for maritime transportation safety and economic viability. However, traditional scheduling methods, primarily based on First Come First Served (FCFS) principles, often produce suboptimal results due to their inability to account for complex spatial–temporal dependencies, directional asymmetries, and varying ship characteristics. This paper introduces SSRL (Ship Scheduling through Reinforcement Learning), a novel framework that addresses these limitations by integrating three complementary components: (1) a Q-learning framework that discovers optimal scheduling policies through environmental interaction rather than predefined rules; (2) a clustering mechanism that reduces the high-dimensional state space by grouping similar ship states; and (3) a sliding window approach that decomposes the scheduling problem into manageable subproblems, enabling real-time decision-making. We evaluated SSRL through extensive experiments using both simulated scenarios and real-world data from the Xiaziliang Restricted Waterway in China. Results demonstrate that SSRL reduces total ship waiting time by 90.6% compared with TSRS, 48.4% compared with FAHP-ES, and 32.6% compared with OSS-SW, with an average reduction of 57.2% across these baseline methods. SSRL maintains superior performance across varying traffic densities and uncertainty conditions, with the optimal information window length of 13–14 ships providing the best balance between solution quality and computational efficiency. Beyond performance improvements, SSRL offers significant practical advantages: it requires minimal computation for online implementation, adapts to dynamic maritime environments without manual reconfiguration, and can potentially be extended to other complex transportation scheduling domains.