Research on detection and location method of safflower filament picking points during the blooming period in unstructured environments

To address the challenges encountered by safflower filament harvesting robots in detecting and localizing harvesting points in unstructured environments, this study proposes a harvesting point detection and localization model based on the DSOE (Detect-Segment-OpenCV Extraction) method, integrated with a localization system using a depth camera. Firstly, the YOLO-SaFi model was employed to optimize the classification of a safflower filament dataset, identifying harvestable safflower filaments for further study. Secondly, a novel lightweight segmentation detection head (LSDH) was introduced, based on the YOLO-SaFi model, to efficiently segment safflower filaments and fruit balls. Using the OpenCV toolkit, contour information of the safflower filaments and fruit balls was extracted. The centroid connection and intersection with the safflower filament contour were used to determine the 2D harvesting points. Finally, a localization control system was developed based on the Delta robotic arm and depth camera to precisely determine the spatial harvesting point locations. Experimental results indicate that the improved YOLO-SaFi-LSDH model reduces the model size by 30.2%, while achieving segmentation accuracy, recall rate, and average precision of 95.0%, 95.0%, and 96.8%, respectively, significantly outperforming conventional detection heads. Additionally, the localization system demonstrated an overall detection success rate of 91.0%, with localization errors controlled within an average of 2.42 mm in the x-axis, 2.86 mm in the y-axis, and 3.18 mm in the z-axis. These results show that the proposed model exhibits superior detection and localization performance in complex environments, providing a solid theoretical foundation for the development of intelligent safflower filament harvesting robots.