Fruit3DGS: Fruit-agnostic counting and localization leveraging semantic-guided 3D Gaussian Splatting and contrastive learning

Fruit counting and localization are essential for yield estimation, crop monitoring, and robotic harvesting. We introduce Fruit3DGS, a fruit-agnostic framework that combines semantic-guided 3D Gaussian Splatting with contrastive instance learning to reconstruct, count, and localize fruits from unordered multi-view RGB images. Existing semantic radiance-field approaches often rely on fruit-specific clustering parameters, predefined size priors, or geometric templates, which limits their transfer across different fruit categories and acquisition conditions. To reduce this dependency, Fruit3DGS augments each Gaussian primitive with a semantic coefficient and a contrastive instance embedding. Prompt-driven binary and instance masks predicted by vision foundation models provide the supervision required to learn foreground semantics and view-consistent fruit identities directly in the 3D representation. Foreground Gaussians are subsequently grouped through a seeded microcluster-merge procedure operating on an embedding-augmented k -nearest-neighbor graph, with its hyperparameters selected automatically through Bayesian optimization. The resulting clusters provide individual fruit instances, predicted counts, and oriented bounding boxes for metric 3D localization without requiring fruit-size templates. We evaluate Fruit3DGS on 14 counting datasets and eight scenes with ground-truth 3D fruit positions. The method remains competitive on established apple-counting benchmarks and transfers more reliably than the original FruitNeRF pipeline to the evaluated non-apple and mixed-fruit scenes. Across the eight localization scenes, Fruit3DGS achieves the lowest mean localization error on seven scenes and an average scene-level mean error of 4.74 cm, compared with 30.29 cm for original FruitNeRF, 56.13 cm for FruitNeRF with an HDBSCAN backend, and 8.49 cm for FruitNeRF++ features clustered with HDBSCAN. This corresponds to a 44.2% reduction in average localization error relative to the strongest competing baseline, FruitNeRF++–HDBSCAN. The results demonstrate that combining semantic-guided Gaussian splatting with contrastive embedding learning enables accurate, fruit-agnostic 3D reconstruction and reliable instance-level perception, while retaining the real-time rendering capability of the optimized 3DGS representation for agricultural robotics and precision-farming applications.

Authors

Institutions

Publication Details

Journal
Computers and Electronics in Agriculture
Published
2026-11-21
DOI
https://doi.org/10.1016/j.compag.2026.112643
Primary Topic
3D Shape Modeling and Analysis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Fruit3DGS: Fruit-agnostic counting and localization leveraging semantic-guided 3D Gaussian Splatting and contrastive learning

Marco Maccarini, Septimius Mara, Loris Roveda, Angelo Moroncelli
Computers and Electronics in Agriculture
3D Shape Modeling and Analysis
article

Fruit3DGS: Fruit-agnostic counting and localization leveraging semantic-guided 3D Gaussian Splatting and contrastive learning

Marco Maccarini, Septimius Mara, Loris Roveda, Angelo Moroncelli
article en

Abstract

Fruit counting and localization are essential for yield estimation, crop monitoring, and robotic harvesting. We introduce Fruit3DGS, a fruit-agnostic framework that combines semantic-guided 3D Gaussian Splatting with contrastive instance learning to reconstruct, count, and localize fruits from unordered multi-view RGB images. Existing semantic radiance-field approaches often rely on fruit-specific clustering parameters, predefined size priors, or geometric templates, which limits their transfer across different fruit categories and acquisition conditions. To reduce this dependency, Fruit3DGS augments each Gaussian primitive with a semantic coefficient and a contrastive instance embedding. Prompt-driven binary and instance masks predicted by vision foundation models provide the supervision required to learn foreground semantics and view-consistent fruit identities directly in the 3D representation. Foreground Gaussians are subsequently grouped through a seeded microcluster-merge procedure operating on an embedding-augmented k -nearest-neighbor graph, with its hyperparameters selected automatically through Bayesian optimization. The resulting clusters provide individual fruit instances, predicted counts, and oriented bounding boxes for metric 3D localization without requiring fruit-size templates. We evaluate Fruit3DGS on 14 counting datasets and eight scenes with ground-truth 3D fruit positions. The method remains competitive on established apple-counting benchmarks and transfers more reliably than the original FruitNeRF pipeline to the evaluated non-apple and mixed-fruit scenes. Across the eight localization scenes, Fruit3DGS achieves the lowest mean localization error on seven scenes and an average scene-level mean error of 4.74 cm, compared with 30.29 cm for original FruitNeRF, 56.13 cm for FruitNeRF with an HDBSCAN backend, and 8.49 cm for FruitNeRF++ features clustered with HDBSCAN. This corresponds to a 44.2% reduction in average localization error relative to the strongest competing baseline, FruitNeRF++–HDBSCAN. The results demonstrate that combining semantic-guided Gaussian splatting with contrastive embedding learning enables accurate, fruit-agnostic 3D reconstruction and reliable instance-level perception, while retaining the real-time rendering capability of the optimized 3DGS representation for agricultural robotics and precision-farming applications.

Computers and Electronics in AgricultureVol. 258
Dalle Molle Institute for Artificial Intelligence Research (CH), Università della Svizzera italiana (CH), Politecnico di Milano (IT)
3D Shape Modeling and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon