Cross-Modal Text Prompts Specified MRI-to-PET Dynamic Image Translation

Motivation: MRI is widely used in clinical settings for its high resolution, but it cannot provide the metabolic information as PET scans. However, PET scans rely on radioactive tracers and pose risks for patients. This study aims to reduce reliance on radioactive tracers by generating PET images from MRI with text prompts. Goal(s): To develop a text-guided MRI-to-PET model using diffusion models, enabling PET synthesis with tracer-specific characteristics. Approach: A diffusion-based model with cross-modal attention was designed, allowing MRI-to-PET generation based on text prompts. Results: The model generated text specified PET images from MRI inputs, with strong similarity to real PET scans. Impact: This approach offers a safer imaging alternative by generating synthesis PET images without radioactive tracers, supporting disease diagnosis and monitoring with reduced patient risk. This development could broaden MRI's clinical application, fostering multi-tracer insights in resource-limited settings.