Diffusion model-based synthesis of brain images for data augmentation

Training robust image segmentation models, particularly for medical applications like brain lesion segmentation, necessitates large, accurately annotated datasets. However, acquiring such datasets is often expensive and time-consuming. To address this challenge, we propose leveraging generative image models, specifically diffusion models, to synthesize realistic MRI brain images. This study investigates the efficacy of two approaches: utilizing ControlNet and employing a custom-designed diffusion model. We evaluate the impact of these synthetic images on U-Net model training in two scenarios: (i) training on real, artificial, and mixed datasets of equal size, and (ii) incrementally augmenting a small real dataset with increasing amounts of synthetic data. We also benchmark the diffusion models against GAN-based competitors to provide a more comprehensive evaluation of their performance. Our findings demonstrate that both diffusion models effectively generate high-quality brain outperforming GANs in this task. Specifically, synthetic data generated by the diffusion models effectively approximates real brain image data, with minimal impact on segmentation performance, with only a 1.5% performance loss observed. Also, it significantly improves the U-Net’s brain lesion segmentation performance when incorporated into the training process. Finally, our findings demonstrate that diffusion-generated images are more effective than GAN-generated images for improving segmentation model training. This research highlights the potential of diffusion models to overcome data limitations in medical image analysis and pave the way for more accurate and efficient lesion segmentation. • The ControlNet and custom diffusion model are proposed to synthesize brain images • Synthetic data generated by diffusion models achieved only 1.5% performance loss • The custom model achieved a small performance gain over ControlNet • For small datasets U-Net model improves lesion segmentation with synthetic data • Diffusion-generated images improve segmentation model training more than GANs