Lora Diffusion: Zero-Shot Lora Synthesis for Diffusion Model Personalization

<p>Low-Rank Adaptation (LoRA) and other parameter-efficient fine-tuning (PEFT) methods provide low-memory, storageefficient<br>solutions for personalizing textto-image models. However, these methods offer little to no improvement in wallclock<br>training time or the number of steps needed for convergence compared to full model fine-tuning. While PEFT methods assume<br>that shifts in generated distributions (from base to fine-tuned models) can be effectively modeled through weight changes in<br>a low-rank subspace, they fail to leverage knowledge of common use cases, which typically focus on capturing specific styles<br>or identities.</p>