HairDiffusion: Designing Your Hair with Diffusion Models

Groundbreaking advancements have emerged within the domain of synthetic image creation over the past decade, with cutting-edge innovations like Stable Diffusion redefining the capabilities of text-guided image generation. However, precise image editing tailored to specific requirements has increasingly garnered attention, with hair editing for facial images emerging as a task of great interest, challenge, and practical value in the field of graphics. This task seeks to restructure hair segments through language-guided instructions, ensuring fidelity to the original image's structural and semantic attributes, ensuring that only the hair is altered in the generated target image, with all other content remaining consistent with the source. Although high-quality and controllable hair editing for facial images achieved through Generative Adversarial Networks (GANs) has demonstrated remarkable results, its applicability and accuracy remain constrained by the inherent limitations of GAN models. To address these challenges, this paper proposes a novel method, HairDiffusion, which leverages large-scale pre-trained diffusion models to significantly enhance the applicability of hair editing for real-world facial images. Through lora fine-tuning, attention map optimization, and a dual-guidance control mechanism with source reference, HairDiffusion achieves diversity and generalization in generated images while effectively preserving the original facial attributes.