Fusing multi-scale attention mechanisms with diffusion models for virtual try-on

Image-based virtual try-on technology aims to transfer a garment from a reference frontal view onto a target human image in a natural and realistic manner. The primary challenge lies in preserving the texture and color details of the reference garment during the transfer, which critically affects the quality of the try-on experience. Recent advancements in diffusion models have shown outstanding performance in various image-related tasks, prompting attempts to apply these models to virtual try-on. However, due to the inherent randomness of diffusion models, maintaining reference consistency, particularly for colors and textures that are difficult to describe linguistically, remains problematic. To address this, a twostage approach to the virtual try-on task is proposed. First, a novel multi-scale attention module is designed to handle natural garment deformation based on appearance flow during the warping process. Subsequently, the generative capabilities of diffusion models are utilized to seamlessly integrate the deformed garment into the target human image. Experimental results indicate that the proposed multi-scale attention module effectively preserves reasonable garment deformation, and when combined with diffusion models, produces superior virtual try-on results.