A Q-Learning-Based Neighborhood Search for Seaside Vehicle Dispatching and Resource Scheduling at Automated Container Terminals

The development of automated container terminals faces the challenge of bridging the gap between the increasing volume of container operations and restricted resources, particularly at the seaside of the terminal. Recognizing the pressing need for effective scheduling and decision-making processes, the industry targets two operational challenges: 1) managing limited apron resources, including vehicles, quay cranes, and traffic lanes; and 2) addressing the operation complexities brought by handling twist-locks. To address this gap, this study proposes a seaside vehicle dispatching and resource scheduling problem, which is formulated as a mixed integer linear programming model. A Q-learning-based neighborhood search is developed along with the proposal of a matheuristic method to obtain high-quality initial solutions. The experiments demonstrate that the proposed approach consistently delivers superior solutions over the benchmarks, significantly advancing the management of automated container terminal operations.