Optimizing Brain Tumor Diagnosis Through Hybrid Deep Learning and Multi-Modal Image Integration

The accurate detection of brain tumors is very important in order to make proper diagnosis and treatment planning. The combination of MRI with CT scans results in multi-modal medical imaging that is complementary and, therefore allows for further analysis. Unfortunately, in conventional fusion methods, information is often lost, and feature representation is suboptimal. In this paper, we introduce a novel hybrid deep learning framework consisting of convolutional neural networks (CNNs) for spatial feature extraction and transformers for addressing long-term dependency. This work proposes a method of fusing multimodal imaging data to optimally improve tumor detection accuracy. The BraTS dataset MRI (T1, T2, FLAIR) and CT images were used for experimental evaluations. Our proposed model yielded the highest per-image accuracy of 93.6%, outperforming both the state-of-the-art U-Net model (85.2%) and PCA-based fusion techniques (78.5%). Moreover, the model was precise (92.4%), recall (91.8%), and f1-score (92.1%), and the Dice similarity coefficient (DSC) was 90.7%. The robustness of the tumor versus non-tumor cases is confirmed using Receiver Operating Characteristic (ROC) analysis, yielding an area under the curve of 0.96. Attention heatmap based qualitative analyses indicated the ability of the model to localize on a tumor region and found the fusion of the most important features from both modalities. The model converged stably within 30 epochs, as backed up by training and validation loss curves. The results of this study demonstrate the promise that hybrid deep learning frameworks hold for furthering multimodal medical imaging and establishing a new benchmark detection rate for brain tumors.