A Monocular Pose Estimation Framework for Automatic Dragon Fruit Harvesting Using Navel and Stem Keypoints

Automated fruit harvesting is crucial for alleviating labor shortages and enhancing agricultural productivity. In this context, it is crucial to obtain information on fruit poses before picking in order to avoid damaging the fruit and/or the plant. However, the complex and unstructured orchard environment poses significant challenges regarding the pose estimation task. In this study, a dragon fruit pose estimation (DFPE) framework using a single RGB image is proposed for dragon fruit automated harvesting, which includes three key components: dataset annotation processing, keypoint detection, and geometric pose estimation. First, a multi-source dataset consisting of 8467 images is constructed to enhance the estimation model’s generalizability. A pseudo four-keypoint annotation strategy is designed to fit the annotation rules of mainstream single-class keypoint detection models and mitigate the inherent limitations of multi-target keypoint detection in agricultural scenarios. This strategy implicitly encodes the fruit’s orientation using bounding box group IDs, while preserving geometric information for pose inference. Then, the fruit body and its two core keypoints (navel and stem) are detected via a real-time keypoint detection model. Notably, the proposed DFPE framework is detector-agnostic: other mainstream keypoint detection models can also be plugged into the subsequent geometric pose inference stage, which guarantees the generality and scalability of the framework. Finally, a dragon fruit pose estimation algorithm based on customized geometric constraints is designed, which takes the detected pose information as the input and outputs the posture of dragon fruit. The results of experiments conducted in natural orchard and laboratory environments demonstrate that the ellipses fitted using the proposed DFPE framework closely aligned with fruit contours, even under foliage occlusion conditions. In the laboratory environment, roll errors reached a maximum of 14.8°, whereas yaw errors peaked at 13.4°. Crucially, all roll and yaw errors remained consistently below 15°, which is well within the tolerance threshold required for non-destructive picking operations using a harvesting robot. In summary, this work presents a low-cost solution for dragon fruit pose estimation from a single RGB image, which can potentially be extended to other ellipsoid crops and is suitable for implementation in harvesting robots operating in orchards.