seg2med: a bridge from artificial anatomy to multimodal medical images

<i>Objective</i>. We present seg2med (segmentation-to-medical images), a modular framework for anatomy-driven multimodal medical image synthesis. The system integrates three components to enable high-fidelity, cross-modality generation of computed tomography (CT) and magnetic resonance (MR) images based on structured anatomical priors.<i>Approach</i>. First, anatomical maps are independently derived from three sources: real patient data, extended cardiac-torso (XCAT) digital phantoms, and anatomies-synthetic subjects created by combining organs from multiple patients. Second, we introduce PhysioSynth, a modality-specific simulator that converts anatomical masks into imaging-like prior volumes using tissue-dependent parameters (e.g. HU,<i>T</i>1,<i>T</i>2,<i>ρ</i>) and modality-specific signal models. It supports simulation of CT and multiple MR sequences, including gradient-echo, SPACE, and volumetric interpolated breath-hold examination. Third, the synthesized anatomical priors are used to train 2-channel conditional denoising diffusion probabilistic models, which take the anatomical prior as a structural condition alongside the noisy image, enabling it to generate high-quality, structurally aligned images within its modality.<i>Main results</i>. The framework achieves a structural similarity index measure (SSIM) of0.94±0.02for CT and0.82±0.12for MR images compared to real patient data, and0.78±0.04FSIM for simulated CT from XCAT. The generative quality is further supported by a Fréchet inception distance of 20.20 for CT synthesis. In modality conversion tasks, seg2med attains SSIM scores of0.91±0.03(MR → CT) and0.77±0.04(CT → MR).<i>Significance</i>. In anatomical fidelity evaluation, synthetic CT images achieve a mean Dice coefficient exceeding 0.90 for 11 key abdominal organs, and over 0.80 for 34 of 59 total organs. These results underscore seg2med's utility in cross-modality image synthesis, dataset augmentation, and anatomy-aware AI development in medical imaging.