Generating MRI Images Based on Clinical Text Using Diffusion Models for Breast Cancer

The synthesis of realistic images from textual descriptions has advanced significantly, particularly through the adoption of diffusion models. These models use probabilistic techniques and iterative refinement to produce high-fidelity visual representations. This research investigates the use of diffusion models in the field of medical imaging, with a particular emphasis on generating MRI scans for breast cancer patients using textual descriptions of their medical conditions. We utilize a text-to-image stable diffusion model and a text-to-video diffusion model to create synthetic MRI images and sequences, respectively. The Duke Breast Cancer MRI dataset, which includes diverse imaging modalities and patient metadata, is used for this purpose. Performance is evaluated using metrics such as Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) for images, and Fréchet Video Distance (FVD) and CLIP similarity scores for videos. Our experiments reveal that text-to-image synthesis with diffusion models can effectively generate high-quality MRI images, with performance influenced by the choice of text tokenization methods, as indicated by a KID metric score of 0.97 for text-to-image models using the CLIP tokenizer. Text-to-video models show potential, with a CLIP similarity metric score of 25.38. These advancements suggest new opportunities for enhancing diagnostic interpretation and augmenting datasets.