Online Teaching: Distilling Decomposed Multimodal Knowledge for Breast Cancer Biomarker Prediction
Immunohistochemical (IHC) biomarker prediction greatly benefits from multimodal data fusion. However, the simultaneous acquisition of genomic and pathological data is often constrained by cost or technical limitations. To address this, we propose a novel Genomics-guided Multimodal Knowledge Decomposition Network (GMKDN), a framework that effectively integrates genomics and pathology data during training while dynamically adapting to available data during inference. GMKDN introduces two key innovations: (1) the Batch-Sample Multimodal Knowledge Decomposition (BMKD) module, which decomposes input features into pathology-specific, modality-general, and genomics-specific components to reduce redundancy and enhance knowledge transferability, and (2) the Online Similarity-Preserving Knowledge Distillation (OSKD) module, which optimizes activation similarity matrices to facilitate robust knowledge transfer between teacher and student models. The BMKD module improves generalization across modalities, while the OSKD module enhances model robustness, particularly when certain modalities are unavailable during inference. Extensive evaluations conducted on the TCGA-BRCA dataset and an external test cohort (QHSU) demonstrate that GMKDN consistently outperforms state-of-the-art (SOTA) slide-based multiple instance learning (MIL) approaches as well as existing multimodal learning models, establishing a new benchmark for breast cancer biomarker prediction. Our code is available at https://github.com/qiyuanzz/GMKDN.
Authors
- Peng Gao (ORCID: https://orcid.org/0000-0002-4721-0887)
- Yanmei Zhu (ORCID: https://orcid.org/0000-0001-5526-7624)
- Q. Zhang (ORCID: https://orcid.org/0009-0000-3835-9498)
- Cheng Lu (ORCID: https://orcid.org/0000-0002-7651-3924)
- Fengyu Cong (ORCID: https://orcid.org/0000-0003-0058-2429)
- Yaqi Du
- Tong Wang (ORCID: https://orcid.org/0000-0002-1998-1537)
- Hongming Xu (ORCID: https://orcid.org/0009-0009-6255-263X)
- Xinyu Hao (ORCID: https://orcid.org/0009-0003-8134-1180)
Institutions
- Dalian University of Technology (CN)
- Dalian University (CN)
- First Hospital of China Medical University (CN)
- Qilu Hospital of Shandong University (CN)
- Southern Medical University (CN)
- China Medical University (CN)
Publication Details
- Journal
- IEEE Transactions on Medical Imaging
- Published
- 2025-01-01
- DOI
- https://doi.org/10.1109/tmi.2025.3628252
- Primary Topic
- AI in cancer detection
- Type
- article
- Field-Weighted Citation Impact
- 0.00