PodNet: Pod real-time instance segmentation in pre-harvest soybean fields

Noninvasive analysis of pod phenotypic traits under field conditions is crucial for soybean breeding research. However, previous pod phenotyping studies focused on postharvest materials or were limited to indoor scenarios, failing to generalize to real-field environments. To address these issues, this paper employs an instance segmentation approach for the precise extraction of the pod area from multiplant RGB images in preharvest soybean fields. We first introduce a cost-effective workflow for constructing datasets of densely planted crop images with a uniform backdrop. Starting with video recording, high-quality static frames are collected by automatic selection. Then, a large vision model is explored to facilitate dense annotation and build a large-scale soybean dataset comprising 20k pod masks. Second, the pod instance segmentation model PodNet is developed based on the YOLOv8 architecture. We propose a novel hierarchical prototype aggregation strategy to fuse multiscale semantic features and a U-EMA prototype generation network to improve the model's perception performance for small objects. Comprehensive experiments suggest that lightweight PodNet achieves a superior mean average accuracy of 0.786 in the custom pod segmentation dataset. PodNet also performs competitively on in-field images without a backdrop and enables real-time inference on the edge computing platform. To the best of our knowledge, PodNet is the first pod instance segmentation model for preharvest fields. The low-cost and high-precision extraction of pods is not only a prerequisite for phenotypic analysis of the pod organs but also constitutes an important foundation in conducting cross-scale phenotyping from whole-plant to seed levels.