AnimeSketchNet: Integrating traditional sketching and anime art via an innovative dataset

Abstract. Recent advancements in text-to-image synthesis have shown promise, yet many models struggle with generalizability and adaptability, particularly in specialized genres like anime sketches. To overcome these challenges, we introduce AnimeSketchNet, a novel framework designed to enhance model performance by effectively capturing the complexity and stylistic variability unique to anime. AnimeSketchNet leverages a diverse collection of anime sketch styles with varying levels of detail, enabling the model to learn and adapt to the distinct artistic style of anime. Our dual-branch architecture, featuring a decoupled attention mechanism, seamlessly assimilates textual descriptions along with visual features, producing high-quality anime sketch that are both detailed and faithful the reference sketch. Experimental results demonstrate that AnimeSketchNet surpasses existing state-of-the-art models in both qualitative and quantitative evaluations. This work significantly advances text-to-image synthesis by offering a powerful and flexible tool tailored specifically to the unique demands of anime art.