<scp>TransGAN</scp>: A Transformer‐<scp>CNN</scp> Mixed Model for Volumetric <scp>CT</scp> to <scp>MRI</scp> Modality Translation and Visualization

ABSTRACT Many clinical procedures necessitate the integration of multi‐modality imaging data to facilitate more informed decision‐making. In practice, the cost of scanning and the potential health risks involved often make the scanning of multi‐modality images impractical. It is therefore important to explore the area of modality translation. In recent years, numerous studies have been conducted with the objective of developing methods for translating images between different modalities. Nevertheless, due to the substantial memory requirements and the difficulty in obtaining perfectly paired data, 3D volume modality translation remains a challenging topic. This research proposes a 3D generative adversarial network for the 3D CT‐MRI modality translation task. In order to leverage both low‐level features (pixel‐wise information) and high‐level features (overall image structure), our method introduces both convolutional and transformer structures. Furthermore, our method demonstrates robustness in the presence of imperfectly paired matched CT and MRI volumes from two medical datasets employed in the research. To validate the network performance, qualitative and quantitative comparisons and ablation studies were conducted. The results of the experiments demonstrate that the proposed framework can achieve good results in comparison to four other methods, with improvements of between 10% and 20% in four objective and one subjective evaluation metrics.