Multi-Agent Reinforcement Learning for Integrated Ship–Port–VTS Coordination in Smart Maritime Transportation Systems

Smart maritime transportation requires coordinated decisions across ship arrival management, port services, and Vessel Traffic Services (VTSs). This study develops a cooperative multi-agent reinforcement learning (MARL) framework representing three interacting functions: aggregate arrival adjustment, port-service response, and traffic metering. The coordination problem is formulated as a cooperative Markov game with normalized operating states, discrete actions, and a shared reward incorporating delay, traffic-risk exposure, emissions, and control effort. The controller implements shared-state cooperative independent Q-learning, with centralized information and agent-specific action selection. Training comprises 5000 episodes of 40 decision steps, followed by frozen-policy evaluation against cross-state rule-based coordination and local-state heuristic control over 400 Monte Carlo episodes per strategy. Relative to these benchmarks, respectively, cooperative MARL reduces mean normalized delay by 44.8% and 50.9%, traffic-risk exposure by 30.6% and 41.2%, and anchorage queue intensity by 41.6% and 37.9%. Absolute cumulative system loss decreases by 12.7% and 24.6%. The emissions proxy is 5.1% lower than under rule-based coordination and 2.3% higher than under local-state heuristic control. The reported training trajectory stabilizes during its final phase, while demand and stochastic-noise tests show sustained comparative advantages in delay, traffic-risk exposure, and queue intensity within the specified environment. These findings describe one trained policy evaluated in an aggregate simulation using assumed transition coefficients and normalized performance proxies. The study provides a computational basis for integrated ship–port–VTS coordination, with operational application requiring field calibration, explicit safety constraints, and further evaluation across training seeds and learning-based benchmarks.