URTOME: An Uncertainty- and Relation-Aware Transformer for 3D Visual Grounding

3D visual grounding aims to localize target objects in point-cloud scenes based on natural-language descriptions. However, incomplete geometry can hinder accurate object-center estimation, insufficient semantic cues can make target discrimination difficult, and similar objects can introduce relational ambiguity. To address these challenges, we propose URTOME, an Uncertainty- and Relation-Aware Transformer that integrates complementary geometric and linguistic modeling. Uncertainty-Aware Voting estimates point-wise voting uncertainty to refine seed-to-center offsets. Multi-Granularity Text Context Enhancement enriches language representations through implicit semantic enrichment and explicit description diversification. The Relation-Aware Transformer models directional and proximity relations among candidate objects to facilitate cross-modal matching. On ScanRefer, URTOME achieves 49.38% [email protected] and 37.32% [email protected], outperforming the 3DVG-Transformer baseline by 4.23 and 4.77 percentage points, respectively. It also achieves overall accuracies of 53.6% and 64.1% on Nr3D and Sr3D, respectively. Ablation studies demonstrate the complementary contributions of the three components, while proposal-level evaluation shows that uncertainty-aware voting improves proposal coverage and reduces vote-center error. These results highlight the effectiveness of integrating voting uncertainty, enriched textual context, and spatial relations for accurate 3D visual grounding.

Authors

Institutions

Publication Details

Journal
ACM Transactions on Multimedia Computing Communications and Applications
Published
2026-11-17
DOI
https://doi.org/10.1145/3863953
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

URTOME: An Uncertainty- and Relation-Aware Transformer for 3D Visual Grounding

Haoran Xie, Dingkun Zhu, Fu Lee Wang, Honghui Fan et al.
3 links
ACM Transactions on Multimedia Computing Communications and Applications
Multimodal Machine Learning Applications
article

URTOME: An Uncertainty- and Relation-Aware Transformer for 3D Visual Grounding

Haoran Xie, Dingkun Zhu, Fu Lee Wang, Honghui Fan, Hechong Li, Libo Zheng, Xinlin Bu
article en

Abstract

3D visual grounding aims to localize target objects in point-cloud scenes based on natural-language descriptions. However, incomplete geometry can hinder accurate object-center estimation, insufficient semantic cues can make target discrimination difficult, and similar objects can introduce relational ambiguity. To address these challenges, we propose URTOME, an Uncertainty- and Relation-Aware Transformer that integrates complementary geometric and linguistic modeling. Uncertainty-Aware Voting estimates point-wise voting uncertainty to refine seed-to-center offsets. Multi-Granularity Text Context Enhancement enriches language representations through implicit semantic enrichment and explicit description diversification. The Relation-Aware Transformer models directional and proximity relations among candidate objects to facilitate cross-modal matching. On ScanRefer, URTOME achieves 49.38% [email protected] and 37.32% [email protected], outperforming the 3DVG-Transformer baseline by 4.23 and 4.77 percentage points, respectively. It also achieves overall accuracies of 53.6% and 64.1% on Nr3D and Sr3D, respectively. Ablation studies demonstrate the complementary contributions of the three components, while proposal-level evaluation shows that uncertainty-aware voting improves proposal coverage and reduces vote-center error. These results highlight the effectiveness of integrating voting uncertainty, enriched textual context, and spatial relations for accurate 3D visual grounding.

ACM Transactions on Multimedia Computing Communications and Applications
Lingnan University (HK), Jiangsu University of Technology (CN), Hong Kong Metropolitan University (HK), Nanjing University of Aeronautics and Astronautics (CN)
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon