Multi-LoRA Composition for Efficiently Learning Fine-Grained Font Styles

Few-shot font generation task aims to generate new font images using the style of a few reference images while preserving the source content. Despite advances in style-content disentanglement, generative networks struggle to handle diverse styles due to their static nature. To address this, a font generation model based on Multi-LoRA composition is proposed, which adaptively synthesizes high-quality fonts using style-specific parameters. The style representations are first extracted using a style encoder. The fonts are then clustered into a predefined number of clusters, and a base font is derived for each cluster. A corresponding LoRA module is optimized for each base font to capture its style feature. During training, the model establishes correlations between the base fonts and their corresponding LoRA modules by learning from all the training fonts. During inference, our proposed model calculates weights based on the similarity between the target and the base fonts, dynamically compositing multiple LoRAs modules to generate the font.