Automated Trimodal Deep Learning Fusion of MRI, CT, and Endoscopy for Preoperative Differentiation of Sinonasal Inverted Papilloma and Malignant Transformation

Purpose: To develop and validate an automated trimodal deep learning fusion framework integrating magnetic resonance imaging (MRI), computed tomography (CT), and nasal endoscopy for preoperative differentiation between sinonasal inverted papilloma (SIP) and SIP-transformed squamous cell carcinoma (SIP-SCC). Materials and Methods: This retrospective study included 849 patients with pathological diagnoses of SIP (n = 614) or SIP-SCC (n = 235) between October 2014 and August 2025. Three unimodal ResNet-based deep learning models were trained and validated using data from 743 patients with only one or two modalities. The trained networks then generated MRI, CT, and endoscopy probability scores for each of 106 separate patients with complete trimodal data. These scores were combined using logistic regression, with fusion performance assessed by five-fold cross-validation in the 106-patient cohort. In the reader study, its performance was compared with six physicians (three junior, three senior). Results: The trimodal fusion model achieved an out-of-fold AUC of 0.910. Its AUC was higher than that of CT (0.634; unadjusted p < 0.001; Holm-adjusted p < 0.001) and endoscopy (0.813; unadjusted p = 0.023; Holm-adjusted p = 0.045), but the difference from MRI (0.859; unadjusted and adjusted p = 0.142) was not statistically significant. At the fixed threshold of 0.5, the fusion model achieved an accuracy of 83.0%. Accuracy differed significantly between the fusion model and the junior-reader group majority vote (64.2%; unadjusted p = 0.004; adjusted p = 0.022), but not the senior-reader group majority vote (81.1%; unadjusted p = 0.860; adjusted p = 1.000). Conclusions: By integrating MRI, CT, and endoscopy images, the trimodal fusion model achieved the highest AUC among the evaluated models for differentiating SIP from SIP-SCC, suggesting its potential value as a clinical decision support tool.