Research on binocular stereo vision phenotyping measurement for leafy vegetable based on 3DGS supervision

Accurate non-destructive measurement of phenotypic data for leafy vegetable is critical for effective management of breeding, growth monitoring, and yield estimation. However, the scarcity of large-scale stereo matching datasets in agricultural scenarios and the high cost of depth-sensing devices tailored for small targets pose significant challenges to achieving accurate phenotyping of crops in complex growing environments. This study introduces a novel 3D Gaussian Splatting (3DGS)-supervised stereo matching network to enable accurate depth estimation of leafy vegetable. We enhance Gaussian rendering techniques to generate stereo training data from multi-view images. A multi-scale feature extraction network with deformable convolutions is proposed to adaptively capture diverse features, ranging from low-level details (edges and textures) to structural and semantic information. To address the multi-peak characteristics of disparity in leafy vegetable, a top- k algorithm is employed for disparity regression, significantly improving disparity estimation accuracy. To enhance the adaptability of cost aggregation, we construct a combined cost volume through a concatenation and group-wise correlations and leverage feature attention blocks to refine disparity prediction. Additionally, depthwise separable convolutions and optimized methods of the number of iterative blocks for leafy vegetable small targets are introduced to reduce computational complexity and accelerate training and inference. Experimental results demonstrate the effectiveness of our depth estimation method, achieving an Endpoint Error (EPE) of 2.32 pixels overall and 0.91 pixels specifically in leafy vegetable area compared to disparity ground truth. Furthermore, we developed a 3D point cloud phenotyping measurement method using instance segmentation and 3D reconstruction. Notably, the measured height, number of leaves, and surface area achieve R 2 values of 0.9946, 0.9577, and 0.9775 against manual measurements, respectively. These high-precision results underscore the practical applicability of our method and provide a theoretical foundation for deployment in agricultural production systems. • Development of an automated leafy vegetable phenotyping system. • Stereo dataset generation via 3D Gaussian Splatting. • Deep learning-based stereo matching for depth estimation. • High-precision phenotyping using 3D point cloud analysis. • Plant factory application of automated phenotyping system.