PlantGaussian: Exploring 3D Gaussian splatting for cross-time, cross-scene, and realistic 3D plant visualization and beyond

Observing plants across time and diverse scenes is critical in uncovering plant growth patterns. Classic methods often struggle to observe or measure plants against complex backgrounds and at different growth stages. This highlights the need for a universal approach capable of providing realistic plant visualizations across time and scene. Here, we introduce PlantGaussian, an approach for generating realistic three-dimensional (3D) visualization for plants across time and scenes. It marks one of the first applications of 3D Gaussian splatting techniques in plant science, achieving high-quality visualization across species and growth stages. By integrating the Segment Anything Model (SAM) and tracking algorithms, PlantGaussian overcomes the limitations of classic Gaussian reconstruction techniques in complex planting environments. A new mesh partitioning technique is employed to convert Gaussian rendering results into measurable plant meshes, offering a methodology for accurate 3D plant morphology phenotyping. To support this approach, PlantGaussian dataset is developed, which includes images of four crop species captured under multiple conditions and growth stages. Using only plant image sequences as input, it computes high-fidelity plant visualization models and 3D meshes for 3D plant morphological phenotyping. Visualization results indicate that most plant models achieve a Peak Signal-to-Noise Ratio (PSNR) exceeding 25, outperforming all models including the original 3D Gaussian Splatting and enhanced NeRF. The mesh results indicate an average relative error of 4% between the calculated values and the true measurements. As a generic 3D digital plant model, PlantGaussian will support expansion of plant phenotype databases, ecological research, and remote expert consultations.