Plant-to-camera enabled 3D morphological reconstruction: A high-fidelity approach for plant phenotyping

Three-dimensional (3D) plant modeling is fundamental for precise phenotyping analysis. In this study, a high-throughput, multi-stage 3D reconstruction pipeline is proposed to efficiently generate point clouds with real-world physical scales. The pipeline comprises five core components: data acquisition, semantic segmentation, sparse reconstruction, dense reconstruction, and phenotypic trait extraction. To enhance the accuracy of plant structure identification, the SegFormer semantic segmentation model is employed for pixel-level segmentation, thereby guiding the subsequent reconstruction stages to focus specifically on plant regions. In feature-based sparse reconstruction, the scarcity of texture information often results in an insufficient number of matching point pairs, leading to failures in camera parameter estimation. Furthermore, the reconstructed point clouds frequently lack consistency with real-world scale. To address these challenges, a calibration-constrained sparse reconstruction method, Sparse Reconstruction from Calibrated Images (SRCI) was proposed. By integrating precise calibration results computed in a custom world coordinate system, SRCI circumvents the limitations of traditional feature matching in scenarios with scarce features, thereby resolving camera pose estimation failures caused by insufficient matching pairs and generating sparse point clouds with true physical scale. Subsequently, CL-MVSNet was employed to generate dense point clouds. The validation experiments are conducted from three perspectives: visual comparison, phenotypic accuracy assessment, and reconstruction accuracy evaluation. First, five groups of rapeseed plants are selected to perform visual comparisons between the proposed reconstruction pipeline and other advanced reconstruction software. The results demonstrate that the proposed reconstruction pipeline achieves superior performance in terms of visual quality. Additionally, three phenotypic parameters of rapeseed plants—plant height, leaf width, and chord length are manually measured and compared with the corresponding phenotypic parameters extracted from the reconstructed point clouds. The analysis revealed mean absolute errors of 4.93 mm, 3.16 mm, and 6.02 mm; root mean square errors of 6.38 mm, 4.56 mm, and 8.35 mm; and coefficients of determination of 0.98, 0.94, and 0.93, respectively. To further validate the generalization performance and accuracy of the proposed method, four additional plant categories with progressively increasing complexity were selected for accuracy evaluation. For the first three plant categories, the Chamfer distances between the reconstructed point clouds and ground truth point clouds were all within 2.4 mm. In the most complex rice reconstruction experiments, the Chamfer distances between the reconstructed point clouds and ground truth point clouds were all within 9 mm, while other methods failed to achieve effective reconstruction. The proposed high-throughput, automated, and high-quality 3D reconstruction framework provides reliable technical support and data resources for genetic research applications, including gene localization, quantitative trait locus analysis, and genome-wide association studies. • We propose a Plant-to-Camera system for high-quality 3D plant reconstruction within 6 minutes, showing strong generalizability. • Our Sparse Reconstruction from Calibrated Images (SRCI) method prevents failures in feature-scarce scenes. • We develop SFNet, a feature descriptor module that fuses multi-frequency to enhance plant feature representation.