Enhancing breast cancer detection in mammographic imaging using explainable clinical decision support system and framework

Breast cancer remains one of the leading causes of mortality among women worldwide, where early and precise detection plays a vital role in improving survival rates and treatment outcomes. However, conventional deep learning approaches often encounter challenges in handling dense mammographic tissues and lack transparency in decision-making, limiting their clinical reliability. To address these limitations, this study introduces TransYOLO-GJO, an explainable and optimized detection framework that integrates transformer-based attention mechanisms into the YOLOv9 architecture and leverages the Golden Jackal Optimization (GJO) algorithm for hyperparameter tuning. The transformer encoder enhances contextual feature extraction, particularly in dense breast regions, while GJO dynamically adjusts critical hyperparameters such as anchor sizes, learning rates, and attention configurations to maximize detection performance. The model was trained and evaluated on the CBIS-DDSM dataset, achieving a mean Average Precision (mAP) of 95.1% and an F1-score of 93.2%, outperforming traditional baseline models. Interpretability was incorporated through Grad-CAM heatmaps and SHAP-based feature attribution, enabling radiologists to visualize the decision-making process. The results confirm that the proposed framework not only enhances diagnostic accuracy but also strengthens trust in AI-assisted clinical decision support systems, making it a promising solution for real-world deployment in breast cancer screening.