Enhanced Brain Cancer Detection Using a Radiomics-Informed Hybrid Autoencoder and GBM Model

Early and correct diagnosis is crucial to improve the survival rate of brain cancer patients. Conventional machine learning classifiers and qualitative radiological assessments are two instances of traditional diagnostic methods that often encounter feature extraction difficulty, possess excessive false positive/negative rates, and cannot deal with intricate spatial and morphological patterns of medical images. This research presents a radiomic strategy combining deep learning with other MRI image-based brain tumour detection methods to overcome these challenges. Our model integrates a strong feature extraction autoencoder with a gradient-boosting machine for classification. MRI images were preprocessed with picture scaling, normalization, and data augmentation from the Brain Cancer Detection MRI Images dataset to enhance the model's generalization. The Hybrid Autoencoder + GBM model outperformed standalone models with 96.8% accuracy, 97.4% precision, 96.2% recall, and 96.8% F1-score.ROC curve studies also validated its effectiveness, demonstrating almost perfect classification with an AUC of 0.99. The proposed hybrid model significantly reduces misclassification errors compared to convolutional neural networks (CNNs), GBM classifiers, and standalone autoencoders. These findings show how hybrid models and deep learning based on radiomics can enhance cancer diagnosis to the level where they can serve as a reliable alternative to traditional methods.