A Unified Diffusion Model for Multimodal Image Reconstruction and Synthesis

Motivation: Multimodal image reconstruction and synthesis are important tasks in medical imaging. Current methods need multiple specialized models, adding complexity to training and deployment. Goal(s): Develop a unified model for both multimodal image reconstruction and synthesis across diverse scenarios, streamlining training and deployment while maintaining consistent performance with specialized models. Approach: Train an unconditional diffusion model on multimodal images and infer it to generate all target modalities from arbitrary combinations of input data. Results: Validated on an MRI-CT-PET dataset, showing the model's capability for multimodal image reconstruction and synthesis with a single pre-trained diffusion model. Impact: This work simplifies multimodality medical imaging by using one model for reconstruction and synthesis, reducing training and deployment complexity. It also offers the potential to enhance clinical imaging efficiency and provides a new tool for leveraging multimodality information.