YOLOv5-Driven Decision Fusion Approach for Aiding Automated Tomato Harvesting: Detection, Maturity Classification, Peduncle Estimation and Picking Point Determination

The first step in automating fruit harvesting entails developing a computer vision system to locate, categorize, and determine the most suitable spot for picking. Tomato detection and picking point estimation typically involve three key components: ascertaining the tomato’s bounding box, categorizing its ripeness, and identifying the optimal picking point. This study proposes a two-part decision fusion approach based on YOLOv5 to locate, classify maturity, and estimate the peduncle position of tomatoes in unstructured environments considering real-world scenarios. In the tomatoes’ identification and maturity classification task, YOLOv5mu achieves an F1 score of $89.4 \%$ and a mAP50 of 0.931. Meanwhile, YOLOv5su achieves a precision of $\mathbf{1 0 0 \%}$ and a recall of $\mathbf{4 8 . 5 \%}$ in estimating peduncle positions, demonstrating enhanced precision compared to the current approaches. Finally, the predictions from these two models are merged to obtain the decision-fused output. The picking points, chosen as the central point of the peduncle estimation boxes, can be used as reference points to direct a robotic arm in facilitating the automated harvesting operation.