An Artificial Intelligence-Based Tool for Enhancing Pectoral Muscle Segmentation in Mammograms: Addressing Class Imbalance and Validation Challenges in Automated Breast Cancer Diagnosis

Breast cancer remains a major health concern worldwide, requiring the advancement of early detection methods to improve prognosis and treatment outcomes. In this sense, mammography is regarded as the gold standard in breast cancer screening and early detection. However, in a scenario where extensive analysis is required, a large set of mammograms conducted by radiologists may carry out false negative or false positive diagnoses. Therefore, artificial intelligence has emerged in recent years as a method for enhancing timing in breast cancer diagnosis. Nonetheless, preprocessing stages are required to prepare the mammography dataset to enhance learning models to correctly identify breast anomalies. In this paper, we introduce a novel method employing convolutional neural networks (CNNs) to segment the pectoral muscle in 1288 mediolateral oblique mammograms (MLOs), thereby addressing class imbalance and overfitting between classes, and dataset augmentation based on translation, rotation, and scale transformation. The effectiveness of the model was assessed through a confusion matrix and performance metrics, highlighting an average Dice coefficient of 0.98 and a Jaccard index of 0.96. The outcomes demonstrate the model capability to accurately identify three classes: pectoral muscle, breast, and background. This study emphasizes the importance of tackling class imbalance problems and augmenting data for the training of models for reliable early breast cancer detection.