Enhanced Brain Tumor Diagnosis Using Transfer Learning with Vision Transformers and Topological Data Analysis

The development of deep learning in brain tumor diagnosis has evolved due to the demand for effective medical image assessment methods. Early attempts utilizing basic machine learning algorithms and convolutional neural networks (CNNs) faced challenges in accuracy, precision, and stability across various datasets. This paper presents the TransTopoDx system, a novel deep learning architecture designed to enhance brain tumor diagnosis by integrating large datasets with topological data analysis (TDA) and vision transformers (ViTs). ViTs, pre-trained with transfer learning on extensive image databases and fine-tuned on specific brain tumor datasets, serve as feature extractors, significantly improving the model’s capacity to discern tumor characteristics. Additionally, TDA is employed using the Mapper algorithm to identify patterns in imaging data and facilitate classification. The feature extraction process involves two parallel paths for ViTs and TDA, with results merged in a fusion layer to enhance tumor prediction accuracy. To optimize weight tuning and mitigate overfitting, dropout techniques and data augmentation is used. The TransTopoDx system, implemented in Python, achieves an impressive overall accuracy of 99.92%, with a Precision of 99.90% and Recall of 99.87%. The incorporation of transfer learning in ViTs training and TDA is crucial for improving diagnostic efficacy.