Enhancing Automated Vending Machine Product Recognition Through Depth-Guided Regression Refinement

Deep neural network advancements have led to significant progress in industrial applications, particularly in product recognition for smart automated vending machines (AVMs). This area has seen increased market demand as a fundamental part of automatedretail. However, the existing works possess critical gaps: 1) densely placed and obscured objects in AVMs lead to inaccurate product recognition results, necessitating auxiliary information for achieving precise detection; 2) the lack of datasets with auxiliary information hinders further development in this field. To address these gaps, we propose a depth-guided product recognition network, which consists of two novel components: a depth-aware feature pyramid network (DFPN) and a depth-aware regression head (DRH). Our DFPN can adaptively select features that are beneficial for regression from both red, green, blue (RGB) and depth data, whereas the DRH refines the regression branch via depth information without affecting the classification process. In addition, to overcome dataset limitations, we develop an extended and fully annotated depth information dataset named <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SmartUVM-D</monospace>, which includes depth information for each image based on the existing SmartUVM dataset. The experimental results obtained on our SmartUVM-D benchmark show that our method effectively solves the inaccurate product recognition problem and achieves substantial gains over the baseline approaches. Specifically, our method (based on the ATSS framework) achieves a mean average precision of 84.4, representing a 2.3-point improvement over the previously developed ATSS method and establishing a new state-of-the-art approach.