Geometry-aware multimodal fusion for large-scale 3D scene understanding

Multimodal 3D semantic segmentation plays a crucial role in engineering applications such as autonomous driving, robotics, and 3D image acquisition and display systems. However, the complex geometry of real scenes, varying object scales, and the inherent sparsity and non-uniform sampling of LiDAR-based acquisition still present significant challenges. To address these issues, we propose an adaptive geometry fusion method for robust and efficient multimodal 3D understanding. The method adopts a dual-path 3D feature extractor: one path captures spatial structural relationships through position-based encoding, while the other integrates a geometry-aware adaptive aggregation module that models local structures using learnable kernel positions and hybrid distance-appearance weighting. Complementary image features from passive optical imaging are fused with LiDAR features to enhance semantic discrimination in regions with sparse or ambiguous 3D measurements. The overall framework remains lightweight, containing only 5.2M parameters, and achieves real-time inference at 25 ms per sample, enabling deployment in resource-constrained sensing and visualization platforms. Experiments on Semantic3D, SemanticKITTI (71.2% mIoU, surpassing the lightweight baseline RandLA-Net by 15.3%), and nuScenes demonstrate competitive performance and strong generalization. Furthermore, evaluations on real-world colored point clouds acquired from a LiDAR-camera system validate the effectiveness of the proposed method in outdoor environments, providing semantically structured 3D representations suitable for downstream 3D display and visualization applications.