Establishment of a Container Ship Loading Plan to Minimize the Number of Re-handles Using DQN

This paper presents a study on the development of an optimal container stowage plan using rein- forcement learning to minimize the number of re-handles. Typically, container stowage plans are manually created by planners who consider factors such as destination, size, type, and weight of the cargo, making the process time-consuming and error-prone. To address this issue, we employed the DQN (Deep Q-N etwork) algorithm, a type of reinforcement learning, to establish an optimal container stowage plan. We proposed a method for defining state using the character- istics of the containers and compared it with traditional methods, using constraints such as desti- nation port and weight. The state definition method proposed in this study has the advantage of not exponentially increasing the number of states with the size of the stowage space by using characteristic values related to the loading status of each row. Initially, the performance of the DQN algorithm was verified using destination port as the only constraint. Subsequently, weight constraints were added to the state definition method that showed better performance. The study results show that the proposed method successfully established a stowage plan that satisfies all constraints in a 4×4 stowage space without causing re-handles, across nine test container lists.