ConsisGaussian: Consistent Object-Level Semantic Features for Enhanced Scene Understanding in 3D Gaussian Splatting

3D Gaussian Splatting has shown exceptional potential in novel view synthesis and 3D reconstruction by leveraging fast rasterization techniques for real-time rendering. Despite its advancements, incorporating semantic information into Gaussians remains challenging due to 2D Object-level Ambiguity and 3D Object-level Ambiguity, significantly impacting the performance of point-level 3D scene understanding and downstream tasks like segmentation and scene editing in 3D space. To address these issues, we propose ConsisGaussian, which ensures all Gaussians corresponding to the same object share a consistent feature. Our method unifies multi-view semantic information, resolving 2D Object-level Ambiguity by classifying objects into a single class and performing class-wise feature fusion. Additionally, we extend the differentiable renderer of 3DGS to create an object-z map, enabling inverse projection to assign consistent features to Gaussians in 3D space and overcome 3D Object-level Ambiguity. Our semantic segmentation and scene editing experiments demonstrate that this approach offers a robust solution for precise 3D scene understanding.