A Unified Approach for Synthesizing Multimodal Brain MR Images via Gated Hybrid Fusion

Motivation: It is possible that some MR images may not be acquired during a scanning session. Therefore, it is necessary to generate missing modalities for accurate diagnosis and treatment planning. Goal(s): Our goal is to synthesize missing modalities from the acquired images while minimizing any loss of information. Approach: We propose a unified framework that employs a gated hybrid fusion approach to synthesize multimodal brain MR images from the acquired images. Results: In our experiments, carried out on the BraTS 2018 database containing four MR modalities, we observed improved synthesis quality in all metrics, when synthesizing missing modalities from one or three provided modalities. Impact: Our unified framework for synthesizing missing modalities is versatile enough to handle all scenarios, irrespective of the modalities provided. It can be applied in clinical decision-making and computer-aided diagnosis, where all modalities are essential for processing.