Key Technology Research on in-situ Apple Recognition and Picking Points Based on Image Processing and Deep Learning

Apples detection in natural environments presents a significant challenge for traditional object detection models due to factors such as occlusion, different lightening conditions, and different backgrounds. As a result, accurately positioning and picking apples using robots becomes difficult. This study intends to improve picking efficiency by providing an integrated technique that uses deep learning and image processing to recognize and find the picking point for red and green apples. A collection of field images including red and green apples is assembled to build testing and training sets. The You Only Look Once v8 (YOLOv8) model is then fine-tuned, with its parameters optimized for recognizing red and green apples and establishing the fruit detection frame's center point. Image processing techniques and a morphological method are utilized to analyze the images by segmenting them, analyzing fruit morphology, extracting the central position of the fruit's outline, and comparing the central point of the YOLO identification frame. The detection performance of various YOLO models, namely YOLOv7 and YOLOv8n, is compared. Among them, the YOLOv8n model exhibits the most promising results. It demonstrates remarkable detection performance and strong generalization abilities for red and green apples in complex scenes. The results include precision, recall, and mean average precision of 91.9%, 82.0%, and 92% for red apples and 92.5%, 84.9%, and 91.6% for green apples. Hence, the combination of YOLOv8n and image processing can effectively detect apples in challenging scenarios. This research provides a useful framework for fruit detection and harvesting under difficult circumstances.