SketchFit: Interactive Diffusion-Driven Fashion Design With Diverse Sketch Adaptation

This paper presents SketchFit, a new framework for user-driven fashion image generation that integrates multi-conditional inputs to guide the learning process. SketchFit combines a Multi-Conditional Adaptive Fusion Strategy (m-CAFs), which unifies diverse input features such as sketches, editable region masks, and incomplete images into a cohesive representation. This representation is further enhanced by a Channel Adaptive Modulator (CAMod) that enables precise feature extraction from grayscale sketches through global pooling and channel-wise attention, ensuring accurate shape completion. The fused features are then mapped into multiple dimensions via a Conditional Fusion mechanism based on Stable Diffusion, improving the ability of the model to handle multi-conditional inputs for personalized fashion design. The framework empowers intuitive, user-guided interactions, enabling high-quality, customizable fashion image generation. Extensive experiments validate the effectiveness of SketchFit in applications ranging from design to customization.