Adaptive Dilated DenseNet and Transformer‐Based Segmentation Model for Early Brain Cancer Detection

ABSTRACT Brain cancer is widely recognized as a severe and complex condition to diagnose and treat. It is detected using imaging techniques like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) scans that play a key role in identifying brain tumors. Additionally, Positron Emission Tomography (PET) scans use a radioactive tracer to monitor metabolic processes like glucose uptake and blood flow, which travel to the brain. In contrast, functional MRI (fMRI) detects changes in blood flow directly without a radioactive tracer by measuring oxygenated versus deoxygenated hemoglobin. A new deep learning framework addresses common challenges in existing 3D deep learning strategies for brain tumor analysis, such as their high computational demands and reliance on extensive, manually annotated data. The new model is purpose‐built to enhance segmentation and classification of brain tumors using 3D MRI images, aiming for improved efficiency and performance. To begin, the required raw 3D MRI images are sourced from standard datasets. The 3D Transformer‐based Dense Atrous Spatial Pyramid Pooling (3D Trans‐DASPP) model enables highly accurate and detailed tumor segmentation. A segmented tumor region is then used as an input for the Double attention‐based Adaptive Recurrent dilated DenseNet (DAReDN) for classification. The DAReDN model's hyperparameters are optimized using a new enhanced fitness‐guided initialization strategy called the Fitness Aspect of Wave Search Algorithm (FtA‐WSA), which is a modification of the standard Wave Search Algorithm (WSA). This FtA‐WSA strategy utilizes a novel random attribute formulation to improve the tuning process, which ultimately leads to enhanced model performance on unseen data. Overall, the model provides an efficient and accurate automated solution for brain tumor analysis from 3D MRI. The 3D Trans‐APP framework has achieved highest accuracy rate of 94.2% by surpassing the existing 85.4% of the Unet framework, 87.6% Unet3+, 87.06% ResUnet, and 87.2% Trans‐Unet achieved in the median measure. Specifically, the proposed model attains 96.65% precision and 0.95% recall to indicate that all positive predictions are correct. This demonstrates a significant improvement in performance over these established methods. On the contrary, the developed method attains better results to showcase the early detection of brain cancer.