Recent advances in multimodal deep learning for breast cancer diagnosis using data fusion vision language models and foundation models

Breast cancer remains one of the most prevalent cancers worldwide, underscoring the need for reliable diagnostic approaches that can support accurate and timely clinical assessment. Recent advances in artificial intelligence, particularly deep learning, have enabled the development of multimodal systems that integrate complementary data sources, including medical images, radiology reports, clinical information, and genomic data. Such approaches aim to exploit information that may not be fully captured by a single modality and have shown promising performance in breast cancer diagnosis and related prediction tasks. This review provides a comprehensive and critical overview of multimodal deep learning approaches for breast cancer diagnosis published between January 2021 and September 2026. It examines the principal data modalities used in multimodal systems, including mammography, ultrasound, magnetic resonance imaging, histopathology, radiology reports, clinical information, and genomic data. The review also analyzes major multimodal fusion strategies and traces the evolution of the field from conventional machine-learning and radiomics-based approaches to deep learning architectures, vision-language models, and foundation models. Recent multimodal models are discussed according to the data modalities employed, model architectures, fusion strategies, clinical or diagnostic tasks, datasets, and reported performance. Particular attention is given to approaches integrating medical imaging with radiology reports and other textual information, including emerging vision-language and foundation-model-based methods for breast imaging. Publicly available datasets and multimodal resources used in breast cancer research are also summarized and compared. Despite recent progress, important challenges remain, including the limited availability of large-scale multimodal datasets, restricted access to paired image–report datasets, missing modalities, limited external validation, uncertainty estimation, explainability, fairness, generalizability, and barriers to clinical workflow integration. Current evidence is still predominantly retrospective and technical; therefore, reported improvements in model performance should not be interpreted as demonstrated clinical benefit without prospective validation. Future research should focus on robust multimodal foundation models, explainable and trustworthy artificial intelligence, federated and privacy-preserving learning, standardized external validation, and prospective clinical evaluation. Overall, this review provides an updated synthesis of methodological developments, multimodal data integration strategies, emerging vision-language and foundation models, and translational challenges in multimodal artificial intelligence for breast cancer diagnosis.

Authors

Institutions

Publication Details

Journal
Discover Artificial Intelligence
Published
2026-10-27
DOI
https://doi.org/10.1007/s44163-026-02625-x
Primary Topic
AI in cancer detection
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Recent advances in multimodal deep learning for breast cancer diagnosis using data fusion vision language models and foundation models

Naoual El Aboudi, Laïla Benhlima
6 links
Discover Artificial Intelligence
AI in cancer detection
article

Recent advances in multimodal deep learning for breast cancer diagnosis using data fusion vision language models and foundation models

Naoual El Aboudi, Laïla Benhlima
article en

Abstract

Breast cancer remains one of the most prevalent cancers worldwide, underscoring the need for reliable diagnostic approaches that can support accurate and timely clinical assessment. Recent advances in artificial intelligence, particularly deep learning, have enabled the development of multimodal systems that integrate complementary data sources, including medical images, radiology reports, clinical information, and genomic data. Such approaches aim to exploit information that may not be fully captured by a single modality and have shown promising performance in breast cancer diagnosis and related prediction tasks. This review provides a comprehensive and critical overview of multimodal deep learning approaches for breast cancer diagnosis published between January 2021 and September 2026. It examines the principal data modalities used in multimodal systems, including mammography, ultrasound, magnetic resonance imaging, histopathology, radiology reports, clinical information, and genomic data. The review also analyzes major multimodal fusion strategies and traces the evolution of the field from conventional machine-learning and radiomics-based approaches to deep learning architectures, vision-language models, and foundation models. Recent multimodal models are discussed according to the data modalities employed, model architectures, fusion strategies, clinical or diagnostic tasks, datasets, and reported performance. Particular attention is given to approaches integrating medical imaging with radiology reports and other textual information, including emerging vision-language and foundation-model-based methods for breast imaging. Publicly available datasets and multimodal resources used in breast cancer research are also summarized and compared. Despite recent progress, important challenges remain, including the limited availability of large-scale multimodal datasets, restricted access to paired image–report datasets, missing modalities, limited external validation, uncertainty estimation, explainability, fairness, generalizability, and barriers to clinical workflow integration. Current evidence is still predominantly retrospective and technical; therefore, reported improvements in model performance should not be interpreted as demonstrated clinical benefit without prospective validation. Future research should focus on robust multimodal foundation models, explainable and trustworthy artificial intelligence, federated and privacy-preserving learning, standardized external validation, and prospective clinical evaluation. Overall, this review provides an updated synthesis of methodological developments, multimodal data integration strategies, emerging vision-language and foundation models, and translational challenges in multimodal artificial intelligence for breast cancer diagnosis.

Discover Artificial IntelligenceVol. 6(1)
Mohammed V University (MA), Ecole Mohammadia d'Ingénieurs (MA), Instituts Supérieurs des Professions Infirmières et Techniques de Santé (MA)
Good health and well-being
Openalex Percentile: Top 14%
AI in cancer detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon