Open-Vocabulary 3D Scene Segmentation via Dual-Modal Interaction

Dear Editor, This letter proposes an innovative open-vocabulary 3D scene understanding model based on visual-language model. By efficiently integrating 3D point cloud data, image data, and text data, our model effectively overcomes the segmentation problem [1], [2] of traditional models dealing with unknown categories [3]. By deeply learning the deep semantic mapping between vision and language, the Network significantly improves its ability to recognize unlabeled categories and exceeds current state-of-the-art methods in the task of scene understanding in open-vocabulary.