Gaussian Semantic Segmentation Based on Color and Shape Deformation Fields

Dynamic scene reconstruction has achieved significant milestones with the advent of 3D Gaussian Splatting (3DGS). However, extending this technology from geometric reconstruction to semantic understanding in dynamic environments remains a challenge. Existing methods often rely on external 2D trackers, which lead to temporal inconsistencies and semantic drift, or suffer from the high computational costs of high-dimensional feature fields. In this paper, we propose a novel framework, Gaussian Semantic Segmentation based on Color and Shape Deformation Fields (GSSBC), to address these issues. Building upon our GBC dynamic scene representation, we bind learnable semantic features to deformable Gaussian primitives. We introduce a spatiotemporal contrastive learning strategy guided by the Segment Anything Model (SAM) to enforce semantic consistency without explicit tracking. Furthermore, we employ a density-based clustering algorithm with label propagation to extract discrete object entities efficiently. Experimental results on the HyperNeRF and Neu3D datasets demonstrate that our method achieves superior segmentation accuracy and spatiotemporal stability compared to state-of-the-art approaches, enabling effective semantic understanding in complex dynamic scenes.