T3DMG: RGB-Based Trackable 3D Model Generation for High-Speed 6-DoF Tracking of Previously Unseen Objects

High-speed six-degree-of-freedom (6-DoF) visual tracking often requires a pre-existing computer-aided design (CAD) model, limiting its use with previously unseen objects. We present Trackable 3D Model Generation (T3DMG), a two-stage framework that uses red–green–blue (RGB) images to acquire a trackable model from two object views before online tracking. The acquisition stage combines foreground-mask refinement, zero-shot 3D generation, and mask-constrained cross-view point-cloud correction. The corrected model is then loaded into Fast-PWP3D for contour-and-texture tracking. On three objects with CAD references, corrected two-view reconstruction increases mean full-angle projection intersection over union from 77.59% to 81.67% relative to direct two-view fusion, and from 73.70% to 81.67% relative to the better single-view setting. Model acquisition takes approximately 100 s typically and 125 s in the reported worst case, before tracker-side model loading. The subsequent online Fast-PWP3D stage reaches up to 420 Hz, with successful rotation tracking up to 600 rpm in the tested setup. Experiments on three additional objects without CAD references demonstrate tracking feasibility. The results support limited-view model acquisition for high-speed visual sensing when pre-tracking acquisition time is acceptable. CAD models serve only as evaluation references.

Authors

Institutions

Publication Details

Journal
Sensors
Published
2026-11-19
DOI
https://doi.org/10.3390/s26227324
Primary Topic
Video Surveillance and Tracking Methods
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

T3DMG: RGB-Based Trackable 3D Model Generation for High-Speed 6-DoF Tracking of Previously Unseen Objects

Daigo FUJIWARA, Satoshi Suzuki, Zejing Zhao, Akio Namiki et al.
Sensors
Video Surveillance and Tracking Methods
article

T3DMG: RGB-Based Trackable 3D Model Generation for High-Speed 6-DoF Tracking of Previously Unseen Objects

Daigo FUJIWARA, Satoshi Suzuki, Zejing Zhao, Akio Namiki, Cheng Ju
article en

Abstract

High-speed six-degree-of-freedom (6-DoF) visual tracking often requires a pre-existing computer-aided design (CAD) model, limiting its use with previously unseen objects. We present Trackable 3D Model Generation (T3DMG), a two-stage framework that uses red–green–blue (RGB) images to acquire a trackable model from two object views before online tracking. The acquisition stage combines foreground-mask refinement, zero-shot 3D generation, and mask-constrained cross-view point-cloud correction. The corrected model is then loaded into Fast-PWP3D for contour-and-texture tracking. On three objects with CAD references, corrected two-view reconstruction increases mean full-angle projection intersection over union from 77.59% to 81.67% relative to direct two-view fusion, and from 73.70% to 81.67% relative to the better single-view setting. Model acquisition takes approximately 100 s typically and 125 s in the reported worst case, before tracker-side model loading. The subsequent online Fast-PWP3D stage reaches up to 420 Hz, with successful rotation tracking up to 600 rpm in the tested setup. Experiments on three additional objects without CAD references demonstrate tracking feasibility. The results support limited-view model acquisition for high-speed visual sensing when pre-tracking acquisition time is acceptable. CAD models serve only as evaluation references.

SensorsVol. 26(22)
Chiba University (JP)
Video Surveillance and Tracking Methods
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon