2.5 dimensional multimodal MRI improves convolutional neural networks performance in predicting the IDH mutation status of adult diffuse gliomas

To the Editor: Adult diffuse gliomas are the most common primary malignant brain tumors and are characterized by high mortality rates.[1] Isocitrate dehydrogenase (IDH), a critical molecular marker, is now being used for the diagnosis and prognostic stratification of adult diffuse gliomas.[2] Traditionally, IDH status has been identified through immunohistochemistry or sequencing techniques. However, these invasive procedures carry certain risks. Magnetic resonance imaging (MRI), a non-invasive imaging technique, can be used to evaluate the IDH mutation status preoperatively. Studies have demonstrated the increasing use of convolutional neural networks (CNNs) in MR-based deep learning for the non-invasive prediction of IDH mutations in glioma.[3] No comprehensive comparison has been made to determine which approach is most effective across large and diverse datasets. The aim of our study was to perform a detailed comparison of the predictive performance of two dimensional (2D), 2.5D and 3D CNN approaches for predicting the IDH status using a large dataset comprising data collected from multiple centers. This study was approved by the Animal and Human Ethics Committee of Beijing Tiantan Hospital, Capital Medical University (No. KY2022-078-02) , and all patients provided their written informed consent. A total of 1889 patients with confirmed adult diffuse gliomas were included, with internal data from Beijing Tiantan Hospital (TT, n = 1151) and external data from The Cancer Genome Atlas (TCGA, external test set 1, n = 237) and the University of California, San Francisco (UCSF, external test set 2, n = 501). The sample size was based on a power calculation, assuming a 5% difference in predictive performance for detecting IDH mutations, with 90% power and a significance level of 0.05, confirming that the study met the design requirements. The inclusion criteria were as follows: (1) histopathologically confirmed primary adult diffuse glioma according to current the World Health Organization (WHO) criteria; (2) known IDH mutation genotype; and (3) available preoperative MR images, including axial T2-weighted (T2w) and postcontrast T1-weighted (T1C) images. The exclusion criteria included missing genetic data or insufficient imaging quality. The inclusion and exclusion criteria for this study are shown in Supplementary Figure 1, https://links.lww.com/CM9/C568. MRI was performed using the 1.5T and 3T scanners from General Electric Company (GE) Healthcare (SIGNA HDe and Discovery 750, USA), Philips (Ingenia CX, Netherlands), and Siemens (Verio, Trio, Prisma, Germany). MRI protocols included T2w and T1C MRI sequences, as presented in Supplementary Table 1, https://links.lww.com/CM9/C568. The images were preprocessed by converting Digital Imaging and Communications in Medicine (DICOM) to Neuroimaging Informatics Technology Initiative (NIFTI) format using SimpleITK (Kitware, USA), skull stripping (BET, FSL, UK), registration (ANTs, USA), and applying N4 bias field correction. To ensure consistency across different scanners and mitigate batch effects, signal intensity normalization was performed using Z-score normalization, which standardizes the intensity values to have a mean of zero and a standard deviation of one. Tumor segmentations were executed automatically using a deep learning algorithm, and the outputs were verified by two neuroradiologists. Three CNN models (2D, 2.5D, and 3D) based on ResNet18 (Microsoft Research, USA) were trained using preprocessed and normalized MRI data. The 2D approach, in which a bounding box referred to the bounding rectangle of the largest tumor in the axial plane, the 2.5D approach, in which three bounding boxes referred to the bounding rectangle of the largest tumor in the axial, sagittal and coronal planes21, and the 3D approach, in which the bounding box referred to the bounding cube of the tumour. Stochastic gradient descent (SGD) with a learning rate of 0.01 and a batch size of 32 was used for 200 epochs to optimize the models. Multi-instance learning was used to predict the mutation rate per slice or volume for each patient’s IDH mutation status. DenseNet121 was used to build the classification model for validation [Supplementary Figure 2, https://links.lww.com/CM9/C568]. Python 3.9 (Python Software Foundation) and R version 4.2 (R Foundation for Statistical Computing, Austria) were used for the statistical analyses. Categorical variables are presented as ratios and were tested using Pearson’s chi-squared test. Continuous variables were tested using the Student’s t-test or the Mann–Whitney U test. The predictive performance of the 2D, 2.5D, and 3D approaches was evaluated using the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). DeLong’s test was used to compare the AUCs for IDH genotype among the approaches and models. Decision curve analysis (DCA) was used to assess the clinical benefits. Statistical significance was set at P <0.05. A total of 1151 patients’ data were collected from the TT dataset, which was divided into a training set (n = 805) and an internal test set (n = 346). External test sets 1 and 2 included data from 237 and 501 patients, respectively. In the training set, the mean age of the patients was 47.7 ± 12.9 years, and the proportion of patients with IDH mutations was 55%. In the internal test set, external test sets 1 and 2, the mean ages were 48.0 ± 12.8 years, 52.3 ± 15.6 years, and 56.9 ± 15.0 years; the proportions of males were 55%, 50%, and 60%; and the percentages of patients with IDH mutations were 54%, 45%, and 21%, respectively [Supplementary Table 2, https://links.lww.com/CM9/C568]. Age has been shown to correlate with IDH mutation status.[4] In our results, we found fewer IDH mutations in the external datasets, which may reflect regional and demographic differences. In addition, the patients in the external test sets were older, which may explain the lower prevalence of IDH mutations. Across all models (ResNet18 and DenseNet121), the performance of the 2.5D approach in predicting IDH genotypes was significantly superior to that of 2D and 3D approaches. In the internal test set and external test sets 1 and 2 using the ResNet18 model, the accuracies of the 2.5D approach were 90.5% (AUC = 0.962), 85.2% (AUC = 0.921), and 84.0% (AUC = 0.900); the accuracies of the 2D approach were 87.0% (AUC = 0.928), 81.4% (AUC = 0.895), and 79.6% (AUC = 0.831); and the accuracies of the 3D approach were 80.9% (AUC = 0.854), 76.4% (AUC = 0.846), and 69.3% (AUC = 0.768), respectively [Supplementary Table 3 and Supplementary Figure 3, https://links.lww.com/CM9/C568]. The DeLong test results showed significant differences between the 2.5D and 2D approaches and between the 2.5D and 3D approaches in the test sets (P <0.01) [Supplementary Table 4, https://links.lww.com/CM9/C568]. Based on the results of the DCA analysis, the 2.5D approach had greater clinical applicability for predicting IDH genotype [Supplementary Figure 4, https://links.lww.com/CM9/C568]. The study also incorporated the DenseNet121 model for comparison [Supplementary Table 5, https://links.lww.com/CM9/C568]. In the internal test set, the AUCs of the 2D, 2.5D, and 3D approaches were 0.921, 0.953, and 0.861, respectively. In external test sets 1 and 2, the AUCs of the 2D, 2.5D, and 3D approaches were 0.854, 0.912, 0.799, 0.795, 0.886, and 0.668, respectively. We found that the 2.5D performed significantly better than both the 2D and 3D approaches in both the ResNet18 and DenseNet121 models. This can be attributed to the ability of the 2.5D approach to incorporate spatial information from three orthogonal planes (axial, sagittal, and coronal). The consistent high performance of the 2.5D approach across various setups in these models highlights its unique architectural advantages and its effective processing of multilevel imaging data by capturing images taken in parallel. Consequently, the 2.5D approach is expected to inspire greater confidence among clinicians, scientists, and others in its application to different scenarios and datasets. Although the 3D approach should theoretically outperform the 2D approach by capturing the entire 3D information of the tumor, our study revealed that the prediction performance of 2D is higher. The reasons for these results may be explained by the following: (1) the 3D approach is computationally complex and prone to overfitting, and (2) different scanning protocols, such as noise and other irrelevant data, affect the quality of 3D images. In contrast, the 2D approach provides a simpler and more robust alternative, achieving in some cases even better diagnostic accuracy, as observed in our study. Future studies should explore the full potential of 3D convolutional neural networks in tumor diagnosis. Among the three approaches, the 2.5D approach achieved the best performance. Age has been shown to correlate with IDH mutation status,[4] therefore, the 2.5D + age model was used to predict the IDH genotype. Supplementary Table 6, https://links.lww.com/CM9/C568 shows the prediction performance of the age-only, 2.5D, and 2.5D + age models in different sets. In the test sets, the results of the age-only model showed accuracies of 69.9% (AUC = 0.750), 76.4% (AUC = 0.816), and 86.2% (AUC = 0.904), respectively. Using the 2.5D model, the accuracies for predicting IDH genotype were 90.5% (AUC = 0.962), 84.8% (AUC = 0.921), and 84.0% (AUC = 0.900), respectively. The 2.5D + age model had accuracies of 91.0% (AUC = 0.958), 87.3% (AUC = 0.940), and 90.8% (AUC = 0.945), respectively. The DeLong test results are shown in Supplementary Table 7, https://links.lww.com/CM9/C568. According to the DCA plots, the 2.5D + age model was associated with a high net benefit [Supplementary Figure 5, https://links.lww.com/CM9/C568]. Compared with the 2.5D model alone, the 2.5D + age model slightly improved the prediction performance. However, there was no statistically significant difference between the 2.5D and 2.5 + age models. This result indicates that the 2.5D model captures most of the information needed to predict the IDH mutation status and that adding age does not provide any meaningful improvement. These findings suggest that the 2.5D model is sufficiently robust to predict IDH mutation status without relying heavily on additional clinical data, making it a valuable non-invasive diagnostic tool. To understand the differences among the three approaches at 1.5T and 3.0T MRI, we evaluated the performance of IDH genotype prediction at different MRI field strengths. In the internal test set, the AUCs of 2D, 2.5D, 3D approaches in 1.5T MRI were 0.947, 0.970, 0.861, respectively, and the AUCs of 2D, 2.5D, 3D approaches in 3.0T MRI were 0.926, 0.959, 0.853, respectively. In the external test set 1, the AUCs of 2D, 2.5D, 3D approaches in 1.5T MRI were 0.899, 0.910, 0.826, respectively, and the AUCs of 2D, 2.5D, 3D approaches in 3.0T MRI were 0.904, 0.924, 0.853, respectively [Supplementary Table 8, https://links.lww.com/CM9/C568]. In the external test set 2, all the data were obtained from 3.0T MRI (GE Discovery 750). The 2.5D model performed well across different MRI field strengths. The robust performance of the 2.5D model at different MRI field strengths demonstrates its generalizability and flexibility in a variety of clinical settings. As MRI scanners may vary from one institution to another, these results show that the 2.5D model can be widely applied in different clinical settings without significant loss of predictive accuracy. Nevertheless, there are certain limitations in this study. First, some advanced sequences, such as perfusion-weighted imaging (PWI), which is becoming increasingly common in clinical settings, may improve approach performance. Second, as a retrospective and multicentre study, there may be selection biases and inherent biases. Therefore, it is necessary to design prospective studies. Finally, the biological mechanisms and clinical interpretations of how model features are related to the IDH genotype remain unclear, and further research is needed to understand these relationships. In conclusion, the 2.5D approach achieved high accuracy in predicting the IDH mutation status in adult diffuse gliomas. Its balance of accuracy and computational efficiency makes it a promising tool for non-invasive IDH status prediction in clinical settings. This approach could improve diagnostic accuracy and assist doctors in making decisions.