Pepper and Object Detection Using Mask R-CNN

Labor shortages and aging populations have accelerated the demand for automated harvesting technologies in agriculture. For robust pepper harvesting, this study proposes a Mask Region-based Convolutional Neural Network (Mask R-CNN) based vision system that can simultaneously detect red peppers, green peppers, and stems at the instance level. To address background–object confusion caused by color similarity between green peppers and leaves, a leaf class is introduced, and a four-class model is compared against the original three-class model under identical training conditions. The four-class model improved the segmentation AP for green pepper and stem classes by 3.6%p and 2.0%p, respectively, while maintaining red pepper detection performance. The proposed system integrates instance segmentation with depth information from an RGB-D camera to estimate three-dimensional cutting positions for robotic harvesting. It achieves an average end-to-end latency of approximately 200 ms and a field-harvesting success rate of 90%, thereby confirming its practical applicability for agricultural harvesting robots.