Apple phenotyping using deep learning and 3D depth analysis: An experimental study on fruitlet sizing during early development

• Achieved high detection accuracy of apple fruitlets in complex orchard environments with rapid phenological changes. • Provided a dataset of videos and RGB-D images, featuring annotated apple fruitlets and manual caliper measurements during early development. • Developed a workflow for rapid in-field monitoring of flower corymbs and fruitlet sizing, validated through experimental trials. Current research in apple-growing focuses on collecting extensive biometric data to better understand physiological processes, improve orchard productivity and predict yields. In this context, fruit thinning has emerged as a key horticultural practice to enhance fruit size and quality while preventing alternate bearing. Despite the growing role of plant imaging technologies in agronomic management, fruitlet sizing remains challenging, particularly in early phenological stages. To address this challenge, we developed an RGB-D-based vision pipeline that combines YOLO models with depth information and relies on the statistical analysis of frame series to detect and cluster fruitlets into flower corymbs, providing both fruitlet counting and diameter estimates for each video acquisition. After obtaining an [email protected] and AP@[0.5:0.95] of respectively 0.894 and 0.77 in fruitlet detection, along with a precision of 0.881 and a recall of 0.846, our approach efficiently processed video frames, extracting the most reliable data for each labeled cluster. While the comparison of true positive estimates with calibrated caliper measurements showed a mean RMSE of 1.05 mm, challenges remain in achieving the correct fruitlet count, with a mean counting error of 0.63 fruitlets per video. Additionally, the proposed workflow retrieved the exact number of fruitlets as the ground truth in 56.4% of the videos, increasing to 75% when excluding those videos where the correct fruitlet count was never detected in any frame by the YOLO model. Despite these limitations, our results are promising, proposing a potential data acquisition tool without compromising the reliability of traditional practices. This approach could pave the way for future applications, including the evaluation of plant growth regulator trials and the development of predictive models for yield and productivity optimization.