3D Localization of Tomato Harvesting Points Based on YOLOv5-Seg

The acquisition of tomato harvesting points is of paramount importance for the realization of automated harvesting. To achieve effective and precise localization of tomatoes in a hanging vine cultivation mode within a greenhouse environment, this study proposes a method for identifying and localizing cherry tomato harvesting points based on binocular vision and deep learning principles. The proposed method, which is built upon tomato pedicel contour segmentation, combines the binocular ranging principle of cameras to accurately obtain the 3D coordinates of cherry tomato pedicel harvesting points in environments. First, YOLOv5-seg is used to detect the contours of the tomato pedicel, followed by skeleton curve fitting using a contour detection algorithm. To address the significant depth loss in tomato pedicels, the depth at both ends of the fitted pedicel contour curve is used to estimate the depth at the midpoint of the curve. The internal and external parameters of the Orbbec Gemini 335 camera are then combined to determine the actual distance between the harvesting point and the camera. Finally, the conversion relationship between the pixel coordinate system and the world coordinate system is applied to obtain the 3D coordinates of the cherry tomato harvesting points. To validate the effectiveness of the proposed method, harvesting point localization experiments for cherry tomatoes were conducted on an optical platform. Through experiments under different positions, heights, and depth distances, the spatial average localization error was found to be 8.24 mm, with an average segmentation speed of 27 ms. The results meet the requirements for fast and precise localization of cherry tomato harvesting points, enabling real-time and accurate segmentation and spatial localization. This study provides a technical reference for the development of cherry tomato harvesting devices.