Incremental Image Generation with Diffusion Models by Label Embedding Initialization and Fusion

Recent diffusion models have excelled in image generation, but adapting them incrementally to new, unseen classes remains difficult. This paper presents LeifDM, an incremental diffusion model that efficiently adapts pre-trained models to new classes in data-scarce environments, minimizing catastrophic forgetting. LeifDM employs a two-stage, classifier-free framework: first, it learns new weights for unseen classes through a fusion strategy, then applies these as initial label embeddings for conditional image generation. This enables LeifDM to produce diverse and realistic images of new classes, even with limited data.