T3DMG: RGB-Based Trackable 3D Model Generation for High-Speed 6-DoF Tracking of Previously Unseen Objects
High-speed six-degree-of-freedom (6-DoF) visual tracking often requires a pre-existing computer-aided design (CAD) model, limiting its use with previously unseen objects. We present Trackable 3D Model Generation (T3DMG), a two-stage framework that uses red–green–blue (RGB) images to acquire a trackable model from two object views before online tracking. The acquisition stage combines foreground-mask refinement, zero-shot 3D generation, and mask-constrained cross-view point-cloud correction. The corrected model is then loaded into Fast-PWP3D for contour-and-texture tracking. On three objects with CAD references, corrected two-view reconstruction increases mean full-angle projection intersection over union from 77.59% to 81.67% relative to direct two-view fusion, and from 73.70% to 81.67% relative to the better single-view setting. Model acquisition takes approximately 100 s typically and 125 s in the reported worst case, before tracker-side model loading. The subsequent online Fast-PWP3D stage reaches up to 420 Hz, with successful rotation tracking up to 600 rpm in the tested setup. Experiments on three additional objects without CAD references demonstrate tracking feasibility. The results support limited-view model acquisition for high-speed visual sensing when pre-tracking acquisition time is acceptable. CAD models serve only as evaluation references.
Authors
- Daigo FUJIWARA (ORCID: https://orcid.org/0009-0005-0691-0122)
- Satoshi Suzuki (ORCID: https://orcid.org/0000-0001-5343-4660)
- Zejing Zhao (ORCID: https://orcid.org/0009-0007-9646-1125)
- Akio Namiki (ORCID: https://orcid.org/0000-0003-3181-0818)
- Cheng Ju (ORCID: https://orcid.org/0000-0002-2282-6917)
Institutions
- Chiba University (JP)
Publication Details
- Journal
- Sensors
- Published
- 2026-11-19
- DOI
- https://doi.org/10.3390/s26227324
- Primary Topic
- Video Surveillance and Tracking Methods
- Type
- article
- Field-Weighted Citation Impact
- 0.00