LIDNeRF

Advances in neural radiance fields (NeRF) have revolutionized the field of 3D scene reconstruction, enabling high-fidelity rendering of complex environments from sparse input data. However, the ability to edit and manipulate such scenes remains a significant challenge. This work introduces an innovative approach to instruction-based image editing by leveraging the capabilities of InstructDiffusion and Lang-SAM models. By combining these models, the system enables precise and context-aware edits to real-world images based on natural language instructions. The core methodology involves an iterative dataset update process where images are rendered from NeRF scenes, updated using diffusion models, and used to supervise scene reconstruction. This approach allows for targeted and localized edits, enabling tasks such as object addition, removal, and replacement while optimizing the underlying 3D scene. The effectiveness of the method is demonstrated through compelling qualitative results, showcasing its versatility and ability to achieve diverse and complex image edits.