Bridging radiology and pathology: domain-generalized cross-modal learning for clinical
Reliable interpretation of clinical imaging requires integrating complementary evidence across modalities, yet most AI systems remain limited by single-modality analysis and poor generalization across institutions. We propose a unified cross-modal framework that bridges mammography and histopathology for breast cancer diagnosis through: (1) a shared vision transformer encoder with lightweight modality-specific adapters, (2) a weakly supervised patient-level contrastive alignment module that learns cross-modal correspondences without pixel-level supervision, (3) domain generalization strategies combining MixStyle augmentation and invariant risk minimization, and (4) causal test-time adaptation for unseen target domains. The model jointly addresses classification, lesion localization, and pathological grading while generating reasoning-guided attention maps that explicitly link suspicious mammographic regions with corresponding histopathological evidence. Evaluated on four public benchmarks (CBIS-DDSM, INbreast, BACH, CAMELYON16/17), the framework consistently outperforms state-of-the-art unimodal, multimodal, and domain generalization baselines, achieving mean AUC of 0.90 under rigorous leave-one-domain-out evaluation and substantially smaller domain gaps (0.03 vs. 0.06-0.10). Visualization and interpretability analyses further confirm that predictions align with clinically meaningful features, supporting transparency and trust. By advancing multimodal integration, cross-institutional robustness, and explainability, this study represents a step toward clinically deployable AI systems for diagnostic decision support.
Authors
- Manimurugan Shanmuganathan
- Hao Sun (ORCID: https://orcid.org/0000-0002-5145-3259)
- Xiang Zhong (ORCID: https://orcid.org/0000-0002-8622-8078)
- Guoqin Jiang
- Meng Li
- Zhuo Gu
- Qian Chen
- Mingming Du
Institutions
- University of Science and Technology of China (CN)
- Soochow University (CN)
- Suzhou Institute of Nano-tech and Nano-bionics (CN)
- Suzhou Municipal Hospital (CN)
- Second Affiliated Hospital of Soochow University (CN)
- University College London (GB)
- University of Tabuk (SA)
Publication Details
- Journal
- npj Digital Medicine
- Published
- 2026-02-16
- DOI
- https://doi.org/10.1038/s41746-026-02423-w
- Primary Topic
- AI in cancer detection
- Type
- article
- Field-Weighted Citation Impact
- 0.00