Reg3D: Reconstructive Geometry Instruction Tuning for 3D Scene Understanding

The rapid development of Large Multimodal Models (LMMs) has led to notable advances in 2D visual understanding, however, extending these capabilities to 3D scene comprehension remains a significant challenge. Existing methods primarily rely on text-only supervision, which lacks the geometric constraints essential for learning accurate and robust 3D spatial representations. To address this limitation, we introduce Reg3D, a novel Reconstructive Geometry Instruction Tuning framework that integrates geometry-aware supervision directly into model training. Unlike previous approaches that inject 3D information solely at the input level, Reg3D adopts a dual-supervision paradigm, utilizing 3D geometry both as input and as explicit training targets. We design complementary object-level and frame-level reconstruction tasks within a dual-encoder architecture to enforce geometric consistency and enhance spatial reasoning. Extensive experiments on four benchmark datasets: ScanQA, Scan2Cap, ScanRefer, and SQA3D demonstrate that with only RGB input, Reg3D outperforms existing methods, establishing a new paradigm for spatially grounded multimodal learning.