L3DM: An Open-Source Large 3D Multi-Modality Model on MindSpore with Text-Geometry Alignment

Abstract We present an efficient and scalable MindSpore-based framework for three-dimensional (3D) scene understanding, namely Large 3D Multi-Modality Model (L3DM), with a focus on text-geometry alignment through MindSpore-native multimodal feature integration and key–value (KV) cache adaptation. Our work delivers three key contributions to the community. First, we provide a dedicated MindSpore performance optimization for multi-modal feature inputs and KV cache, resolving prior bottlenecks and enabling stable, high-performance operation where it was previously challenging. Second, we explore zero-shot Chain-of-Thought reasoning for 3D multi-view understanding, providing valuable insights into its limitations for complex spatial reasoning tasks. Third and foremost, we offer the first complete, open-source, and scalable 3D foundation model implementation on the MindSpore platform. This contribution fills a critical gap by providing the community with a fully functional training and inference codebase, pre-trained weights, and pipelines for a multimodal 3D large model—resources that were entirely absent before. Overall, the proposed L3DM method establishes a new, stable, and accurate benchmark solution for 3D understanding on MindSpore, with impressive empirical validation on the ScanQA benchmark. Code is available at https://openi.pcl.ac.cn/OpenAIRobotics/L3DM.