Early Stage Breast Cancer Detection With Customized <scp>GAN</scp> and Multiview Transformer Model in Mammography Images

ABSTRACT Breast cancer is considered one of the most predominant cancers among women worldwide. Thus, it is crucial for early and accurate diagnosis to make effective treatment and improved diagnosis of patient outcomes. Nowadays, deep learning‐based approaches have revolutionized the field of medical imaging by providing high accuracy solutions through layered structures. However, existing deep learning‐based breast cancer classification models face challenges such as inaccurate disease classification and the unavailability of large datasets necessary for making the diagnosis more effective. Thus, this research introduced a deep learning‐based data augmentation and classification model for enhancing the accuracy of disease detection. Initially, the mammogram images are collected from publicly available and real datasets. To increase the number of images, the proposed approach uses a customized generative adversarial network (Cust‐GAN) for data augmentation. To increase the quality and remove noise in the image, adaptive bilateral filters with weight (ABFW) were used for image pre‐processing. A novel attention‐based multi‐view swin patch transformer (Att‐MVSPtrans) model is introduced for early breast cancer classification. The proposed model achieved accuracies of 99.69% on the CBIS‐DDSM dataset, 99.88% on the MIAS dataset, and 97.2% on the real dataset, consistently outperforming the compared methods.