Efficient latent-vision enabled generative data augmented pipeline for medical image analysis with hybrid deep learning approach

Medical image analysis is an important process involved in diagnosing diseases, but a lack of data, class imbalance, and poor interpretability of deep learning models hamper it. In this study, these problems are overcome by proposing a hybrid generative data-augmented method for the classification of breast cancer and brain tumors using Vision Transformer (ViT) based learning. The proposed method uses Latent Diffusion Model (LDM)-based augmentation, EfficientNetV2 for extracting features from medical images, selection of important features through the Attention-based Feature Selection (AFS) technique, and use of Transformer networks for classification. The proposed model is evaluated by applying it to two datasets, CBIS (Curated Breast Imaging Subset of DDSM)- Digital Database for Screening Mammography (DDSM) and Brain Tumor Segmentation (BraTS2020), using a leakage-free patient-wise split method and a multi-run validation process. The experimental results show that the proposed method achieves excellent performance with 96.54% accuracy on BraTS and 96.23% on CBIS-DDSM, and F1 scores of 96.40% and 96.11%, respectively, along with Receiver Operating Characteristic – Area Under the Curve (ROC AUC) scores of 0.999 for both datasets. Quality of the augmentation process can be validated through the Structural Similarity Index Measure (SSIM) score that ranges between 0.82 and 0.91, as well as through the Dice score ranging between 0.78 and 0.89, which means structural information is successfully preserved during the training process. Statistical validation, it can confirm low variance within the dataset as well as significant improvements achieved when compared to state-of-the-art models. Incorporation of interpretability with the help of Gradient-weighted Class Activation Mapping (Grad-CAM) contributes towards increased clinical relevancy by identifying tumor areas.