185 Development and Validation of an Unpaired Multi-Modality Image Generation Model on Brain Tumor Segmentation in Sequence Missing Scenarios

INTRODUCTION: Missing MRI sequences due to limited scan time, image artifacts, scan corruption, allergies to contrast agents, etc. in real clinical environments limit the availability of deep learning (DL) brain tumor segmentation models. Currently, there is no systematic evaluation of the impact of missing sequences on DL-based brain tumor segmentation models, nor is there a unified approach to address these diverse scenarios. METHODS: We proposed a pipeline for brain tumor segmentation in various MRI sequence missing scenarios. A novel unpaired multi-modal generative adversarial transformer (UMMGAT) was designed to perform image-to-image translation among MRI sequences from the BRATS dataset (335 cases) and local dataset (92 cases). The generated images were then used to replace missing ones as inputs for a multi-modal segmentation network. RESULTS: The UMMGAT can be effectively trained with unpaired data to perform arbitrary image-to-image translations among multi-center multi-sequence MR images. The median DSCs of the brain tumor segmentation are significantly improved by using generated images vs copied images under most of the scenarios. Notably, in scenarios where Flair, T1, T2, T1ce, Flair&T1ce, and T1&T1ce sequences were absent, using gFlair_from_T2 (generated Flair from T2), gT1_from_T2, gT2_from_T1, gT1ce_from_T1, gFlair_from_T2 & gT1ce_from_T1, and gT1_from_T2 & gT1ce_from_T2 vs using copied source sequences for substitution, the median DSCs of WT were 0.734(0.591 - 0.832) vs 0.554(0.341 - 0.689) , 0.905(0.774 - 0.927) vs 0.854(0.704 - 0.911) , 0.865(0.742 - 0.908) vs 0.781(0.515 - 0.888) , 0.894(0.794 - 0.917) vs 0.849(0.687 - 0.914) , 0.71(0.536 - 0.81) vs 0.436(0.144 - 0.615) , and 0.897(0.756 - 0.919) vs 0.721(0.512 - 0.858)(p<0.0001). CONCLUSIONS: The proposed UMMGAT can synthesize high-fidelity images through training on unpaired datasets. The generated images exhibit potential to enhance the DL brain tumor segmentation model in simulated sequence missing scenarios.