Object Detection for Product Arrangement Robot Using Anomaly Detection

The product arrangement robot, which displays products at retail stores, is one of the applications of industrial robot arms. For automatic product arrangement using robot, it is necessary to detect the products to be arranged. Because the products to be arranged can exist in countless states, it is difficult to define all states of products to be arranged. Therefore, in this study, we focus on the fact that the products to be arranged are in an anomaly state and consider the use of anomaly detection. However, there are two problems associated with product detection using anomaly detection. First, the anomaly area estimated using anomaly detection does not correspond to the product area, which is ambiguous. Second, the anomaly product area that can be detected depends on the sensors used for the anomaly detection. For the first problem, we utilize the segmentation foundation model “Segment Anything.” Using the coordinates calculated based on the anomaly area as a prompt, it is possible to accurately extract only the products to be arranged. For the second problem, we define a new third state, “indetermination” in addition to normal and anomaly. By selecting the anomaly detection neural network (NN) that is not in an indetermination from among multiple NN, products to be arranged can be correctly detected. The comparison results of the single anomaly detection NN and our proposed method showed that the detection accuracy of the product areas improved from 46.5% to 71.6%. Furthermore, a robot using the proposed method successfully picked the products to be arranged up from the shelves.