EACNet: Ensemble adversarial co-training neural network for handling missing modalities in MRI images for brain tumor segmentation

Brain tumor segmentation is critical in clinical diagnosis and treatment planning. Existing methods for brain tumor segmentation with missing modalities often struggle when dealing with multiple missing modalities, a common scenario in real-world clinical settings. These methods primarily focus on handling a single missing modality at a time, making them insufficiently robust for the additional complexity encountered with incomplete data containing various missing modality combinations. Additionally, most existing methods rely on single models, which may limit their performance and increase the risk of overfitting the training data. This work proposes a novel method called the ensemble adversarial co-training neural network (EACNet) for accurate brain tumor segmentation from multi-modal magnetic resonance imaging (MRI) scans with multiple missing modalities. The proposed method consists of three key modules: the ensemble of pre-trained models, which captures diverse feature representations from the MRI data by employing an ensemble of pre-trained models; adversarial learning, which leverages a competitive training approach involving two models; a generator model, which creates realistic missing data, while sub-networks acting as discriminators learn to distinguish real data from the generated “fake” data. Co-training framework utilizes the information extracted by the multimodal path (trained on complete scans) to guide the learning process in the path handling missing modalities. The model potentially compensates for missing information through co-training interactions by exploiting the relationships between available modalities and the tumor segmentation task. EACNet was evaluated on the BraTS2018 and BraTS2020 challenge datasets and achieved state-of-the-art and competitive performance respectively. Notably, the segmentation results for the whole tumor (WT) dice similarity coefficient (DSC) reached 89.27%, surpassing the performance of existing methods. The analysis suggests that the ensemble approach offers potential benefits, and the adversarial co-training contributes to the increased robustness and accuracy of EACNet for brain tumor segmentation of MRI scans with missing modalities. The experimental results show that EACNet has promising results for the task of brain tumor segmentation of MRI scans with missing modalities and is a better candidate for real-world clinical applications.