EDG‐CDM: A New Encoder‐Guided Conditional Diffusion Model‐Based Image Synthesis Method for Limited Data

ABSTRACT The Diffusion Probabilistic Model (DM) has emerged as a powerful generative model in the field of image synthesis, capable of producing high‐quality and realistic images. However, training DM requires a large and diverse dataset, which can be challenging to obtain. This limitation weakens the model's generalisation and robustness when training data is limited. To address this issue, EDG‐CDM, an innovative encoder‐guided conditional diffusion model was proposed for image synthesis with limited data. Firstly, the authors pre‐train the encoder by introducing noise to capture the distribution of image features and generate the condition vector through contrastive learning and KL divergence. Next, the encoder undergoes further training with classification to integrate image class information, providing more favourable and versatile conditions for the diffusion model. Subsequently, the encoder is connected to the diffusion model, which is trained using all available data with encoder‐provided conditions. Finally, the authors evaluate EDG‐CDM on various public datasets with limited data, conducting extensive experiments and comparing our results with state‐of‐the‐art methods using metrics such as Fréchet Inception Distance and Inception Score. Our experiments demonstrate that EDG‐CDM outperforms existing models by consistently achieving the lowest FID scores and the highest IS scores, highlighting its effectiveness in generating high‐quality and diverse images with limited training data. These results underscore the significance of EDG‐CDM in advancing image synthesis techniques under data‐constrained scenarios.