FocusSculpt: Part-Level Modification of 3D Point Clouds Through Text and Visual Prompts

Text-prompt-based editing of 3D objects is becoming crucial in diverse areas, including immersive environments like VR, gaming, and digital education, where the ability to create tailored, interactive content is highly valued. However, due to the inherent limitations of textual descriptions, current approaches still face challenges in accurately managing the appearance and location of the edited outputs. FocusSculp addresses these challenges through an innovative framework designed to edit 3D objects with high precision using text and image prompts. It employs a 2D personalized approach to better understand the concept of the reference images and uses zero-shot part segmentation to align text-driven modifications on point clouds accurately. The framework incorporates 3D Gaussian splatting, a flexible and robust form of 3D representation, optimized through both local and global score distillation sampling to ensure edits are confined to specified areas without affecting the overall structure. Enhancements include position embeddings and a deform network that adaptively handles transformations during the editing process. Extensive experiments shows that FocusSculpt effectively executes precise edits and maintains the realism of untargeted sections, enhancing the text-to-3D object editing capabilities.