SceneReasoner: Decoupled Spatial Tokenization for Indoor Large-Scene Understanding with LLMs

Abstract. Most existing 3D vision-language models focus on object-level or single-room understanding and perform poorly in large-scale, multi-room indoor environments where task-relevant objects constitute only a small fraction of the total point cloud. When multi-room point clouds are fed directly into an LLM, critical semantic signals are diluted by the vast amount of redundant background, making it difficult for the model to focus on truly relevant regions. We propose SceneReasoner, a decoupled spatial tokenisation framework that addresses this challenge through three core designs: (1) pre-tokenisation text-guided feature weighting that leverages the shared CLIP embedding space between OpenScene point features and text queries to amplify question-relevant point features before any spatial compression occurs; (2) 2D–3D feature fusion that integrates top-down 2D CLIP features with 3D sparse tokens, supplying the model with appearance semantics—such as texture, material, and room layout—absent from raw point clouds; and (3) layer-wise dense feature injection that inserts local dense features into the LLM attention mechanism layer by layer for fine-grained perception of key regions. We evaluate on the XR-Scene benchmark, which covers cross-room question answering and scene captioning over HM3D indoor environments with an average area of 132 m2. SceneReasoner achieves the best CIDEr on XR-SceneCaption (+0.33 over LSceneLLM), the highest METEOR on XR-QA, and competitive ROUGE-L across all three tasks, demonstrating the effectiveness of task-guided spatial tokenisation for large-scene understanding.