Real-time twin automated double cantilever rail crane scheduling problem for the U-shaped automated container terminal using deep reinforcement learning

The real-time twin automated double cantilever rail crane scheduling problem (RADCRCSP) is crucial for the newly proposed U-shaped automated container terminal (ACT) to improve operation efficiency and minimize energy consumption. However, the complexities and dynamics inherent in the system introduce significant challenges in solving the RADCRCSP. To address the challenge, this study proposes a deep reinforcement learning (DRL) based scheduling model for the RADCRCSP by formulating it as a Markov decision process (MDP) and designing novel rewards, efficient actions, and relevant state features. Then, a DRL-based real-time scheduling optimization (DRL-RSO) framework is developed to solve the RADCRCSP. This framework integrates the double deep Q-network (DDQN) algorithm with simulation techniques to learn optimal mixed rule policies (OMRPs) for task selection in the simulated dynamic complex environment of the twin ADCRCs operation. Experimental results demonstrate that the OMRPs outperform traditional heuristic scheduling rules for import and export blocks in the U-shaped ACT, demonstrating the effectiveness of the DRL-RSO framework. The findings offer decision support for enhancing efficiency and reducing energy consumption of the U-shaped ACT.