URTOME: An Uncertainty- and Relation-Aware Transformer for 3D Visual Grounding
3D visual grounding aims to localize target objects in point-cloud scenes based on natural-language descriptions. However, incomplete geometry can hinder accurate object-center estimation, insufficient semantic cues can make target discrimination difficult, and similar objects can introduce relational ambiguity. To address these challenges, we propose URTOME, an Uncertainty- and Relation-Aware Transformer that integrates complementary geometric and linguistic modeling. Uncertainty-Aware Voting estimates point-wise voting uncertainty to refine seed-to-center offsets. Multi-Granularity Text Context Enhancement enriches language representations through implicit semantic enrichment and explicit description diversification. The Relation-Aware Transformer models directional and proximity relations among candidate objects to facilitate cross-modal matching. On ScanRefer, URTOME achieves 49.38% [email protected] and 37.32% [email protected], outperforming the 3DVG-Transformer baseline by 4.23 and 4.77 percentage points, respectively. It also achieves overall accuracies of 53.6% and 64.1% on Nr3D and Sr3D, respectively. Ablation studies demonstrate the complementary contributions of the three components, while proposal-level evaluation shows that uncertainty-aware voting improves proposal coverage and reduces vote-center error. These results highlight the effectiveness of integrating voting uncertainty, enriched textual context, and spatial relations for accurate 3D visual grounding.
Authors
- Haoran Xie (ORCID: https://orcid.org/0000-0003-0965-3617)
- Dingkun Zhu (ORCID: https://orcid.org/0000-0003-2447-4828)
- Fu Lee Wang (ORCID: https://orcid.org/0000-0002-3976-0053)
- Honghui Fan (ORCID: https://orcid.org/0000-0003-4561-5853)
- Hechong Li (ORCID: https://orcid.org/0000-0001-6842-3446)
- Libo Zheng (ORCID: https://orcid.org/0009-0000-1505-1028)
- Xinlin Bu (ORCID: https://orcid.org/0009-0007-8418-7624)
Institutions
- Lingnan University (HK)
- Jiangsu University of Technology (CN)
- Hong Kong Metropolitan University (HK)
- Nanjing University of Aeronautics and Astronautics (CN)
Publication Details
- Journal
- ACM Transactions on Multimedia Computing Communications and Applications
- Published
- 2026-11-17
- DOI
- https://doi.org/10.1145/3863953
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00