3D Vision-Based Perception and Length Estimation of Green Asparagus for Selective Harvesting

Highlights Different 3D imaging sensors were evaluated indoors for green asparagus detection and length estimation. A database of asparagus field images with 1008 length measurements of spears in the field was established. Among three small-scale YOLO models (v8, v9, and v10), YOLOv8s achieved the best asparagus detection with an F1 score of 0.85. The perception pipeline achieved a spear length estimation accuracy of 89.4%, a base point localization error of 1.4 cm, and a detection rate of 95.8% of harvestable spears with only a spear length criterion applied. The entire algorithm pipeline achieved 3.3 frames per second for green asparagus perception onboard. ABSTRACT. Green asparagus is a labor-intensive vegetable crop for harvesting. Rising labor costs and seasonal labor shortages are threatening the sustainability and profitability of the asparagus industry in the United States (U.S.), making harvesting automation more urgently needed than ever. A multitude of challenges, however, exist in developing practically variable automated harvesting technology for green asparagus. This study describes a novel effort aimed at the development and evaluation of a 3D (three-dimensional) vision-based asparagus perception system for selective harvesting of U.S. green asparagus. Three different types of 3D cameras were evaluated indoors for asparagus detection and length estimation. A time-of-flight technology-based camera yielded the best accuracy and was deployed on a mobile vision system for imaging asparagus in diverse field conditions. A new annotated dataset alongside ground-truth length measurements for 1008 spears was created for asparagus perception algorithm development and evaluation. Among three small-scale YOLO (v8, v9, and v10) models, YOLOv8s achieved the best F1 score of 0.849. The YOLOv8s-based pipeline point cloud processing algorithms, including segmentation, clustering, and outlier removal, achieved a percentage accuracy of 89.4% (with the corresponding error of about 2.3 cm) in spear length estimation and the averaged base point localization error of 1.4 cm. The entire algorithm pipeline for spear detection and localization could be run at about 3.3 frames per second on an onboard computer. The harvest eligibility analysis of detected spears showed a detection rate of 95.8% of harvestable spears when only the length criterion (greater than 8 inches or 20.3 cm) was applied and a detection rate of 84.5% when both the length and base point localization (error less than 2 cm) criteria were considered. The 3D vision-based perception system in this study is promising for reliable and real-time asparagus perception in production fields. Keywords: Artificial intelligence, Green asparagus, Machine vision, Point cloud processing, Selective harvesting.