Online 3D Bin Packing Using a Combination of Tree Structures and Deep Reinforcement Learning

The online 3D bin packing problem (online 3D-BPP) is crucial for optimizing packing efficiency in various industrial applications. In this work, we introduce two external areas: an item buffer for the selection of packing items, and a temporary backup space to allow the rearrangement of items already packed in the bin. We propose a novel method that combines a deep reinforcement learning-based placement position planner with a movement sequence planner structured as a tree with the use of these external areas. This approach enables efficient planning of item placement and movement sequences without expanding the learning space, regardless of the size of external areas. Through numerical experiments, our method demonstrates a consistent improvement in spatial filling rates and robustness against constraints like physical stability. Notably, by utilizing these external areas, we elevated the performance of existing DRL methods by approximately 15 % in the scenario involving complex constraints such as physical stability. These results suggest that our approach is highly effective for complex, real-time packing scenarios in logistics and warehousing.