Enhancing Zero-Shot Learning with CLIP for Multi-Distribution Mammogram Classification

Breast cancer, the most prevalent cancer among women globally, relies on mammography for early detection, significantly reducing mortality rates. The Breast Imaging Reporting and Data System (BI-RADS) standardizes classification of mammograms into normal, benign, or malignant categories, aiding diagnosis. However, existing vision-language models (VLMs) struggle with generalization across diverse datasets due to imaging variations. This thesis introduces the Multi-Distribution Mammogram Classification with Contrastive Language-Image Pre-training (MDMC-CLIP) framework, enhancing the OpenAI CLIP model to improve accuracy and cross-dataset generalization in mammography. By integrating multi-dataset training (INbreast, MIAS, VinDr-Mammo, KAU-BCMD, CMMD) and enriched prompts, MDMC-CLIP captures dataset-specific diversity, enabling differentiation of subtle inter-dataset variations alongside standard classifications. Latent prompt extraction refines image descriptions, boosting fine-grained understanding. Zero-shot experiments demonstrate MDMC-CLIP’s superiority over baseline VLMs, showcasing enhanced accuracy and adaptability to unseen datasets. This research advances AI-driven mammography, offering a robust tool for early breast cancer detection with broad clinical potential.