Fruit3DGS: Fruit-agnostic counting and localization leveraging semantic-guided 3D Gaussian Splatting and contrastive learning
Fruit counting and localization are essential for yield estimation, crop monitoring, and robotic harvesting. We introduce Fruit3DGS, a fruit-agnostic framework that combines semantic-guided 3D Gaussian Splatting with contrastive instance learning to reconstruct, count, and localize fruits from unordered multi-view RGB images. Existing semantic radiance-field approaches often rely on fruit-specific clustering parameters, predefined size priors, or geometric templates, which limits their transfer across different fruit categories and acquisition conditions. To reduce this dependency, Fruit3DGS augments each Gaussian primitive with a semantic coefficient and a contrastive instance embedding. Prompt-driven binary and instance masks predicted by vision foundation models provide the supervision required to learn foreground semantics and view-consistent fruit identities directly in the 3D representation. Foreground Gaussians are subsequently grouped through a seeded microcluster-merge procedure operating on an embedding-augmented k -nearest-neighbor graph, with its hyperparameters selected automatically through Bayesian optimization. The resulting clusters provide individual fruit instances, predicted counts, and oriented bounding boxes for metric 3D localization without requiring fruit-size templates. We evaluate Fruit3DGS on 14 counting datasets and eight scenes with ground-truth 3D fruit positions. The method remains competitive on established apple-counting benchmarks and transfers more reliably than the original FruitNeRF pipeline to the evaluated non-apple and mixed-fruit scenes. Across the eight localization scenes, Fruit3DGS achieves the lowest mean localization error on seven scenes and an average scene-level mean error of 4.74 cm, compared with 30.29 cm for original FruitNeRF, 56.13 cm for FruitNeRF with an HDBSCAN backend, and 8.49 cm for FruitNeRF++ features clustered with HDBSCAN. This corresponds to a 44.2% reduction in average localization error relative to the strongest competing baseline, FruitNeRF++–HDBSCAN. The results demonstrate that combining semantic-guided Gaussian splatting with contrastive embedding learning enables accurate, fruit-agnostic 3D reconstruction and reliable instance-level perception, while retaining the real-time rendering capability of the optimized 3DGS representation for agricultural robotics and precision-farming applications.
Authors
- Marco Maccarini (ORCID: https://orcid.org/0000-0002-9081-7377)
- Septimius Mara (ORCID: https://orcid.org/0000-0002-5252-8347)
- Loris Roveda (ORCID: https://orcid.org/0000-0002-4427-536X)
- Angelo Moroncelli (ORCID: https://orcid.org/0009-0002-2100-5225)
Institutions
- Dalle Molle Institute for Artificial Intelligence Research (CH)
- Università della Svizzera italiana (CH)
- Politecnico di Milano (IT)
Publication Details
- Journal
- Computers and Electronics in Agriculture
- Published
- 2026-11-21
- DOI
- https://doi.org/10.1016/j.compag.2026.112643
- Primary Topic
- 3D Shape Modeling and Analysis
- Type
- article
- Field-Weighted Citation Impact
- 0.00