Reinforcement learning approach for outbound container stacking in container terminals

The container stacking problem for outbound containers is a major planning task under yard operations in a container terminal. It is important to minimize the expected number of rehandling operations, which further helps maintain the productivity of yard operations and improve the efficiency of container terminals. In this paper, we propose a reinforcement learning based approach to determine the storage location of the arriving outbound containers. A reinforcement learning approach is developed to identify the appropriate storage location, aiming to minimize the expected number of rehandling operations during the loading operation. Furthermore, we developed suitable strategies related to reinforcement learning to determine the storage location by training the model using a sufficient number of episodes. Numerical experiments were conducted to compare the proposed model with existing algorithms using real-life container terminal data. The experimental results indicate that the proposed model is robust to uncertain environments, supports real-time decisions, and minimizes the expected number of rehandling operations. • A novel reinforcement learning is proposed to address container stacking problem. • Monte Carlo Q-learning model is developed. • A state is proposed to represent container stacking process efficiently. • A novel reward function is proposed to reduce the possibility of future rehandling. • The proposed method improves the existing online and offline algorithms.