Online Teaching: Distilling Decomposed Multimodal Knowledge for Breast Cancer Biomarker Prediction

Immunohistochemical (IHC) biomarker prediction greatly benefits from multimodal data fusion. However, the simultaneous acquisition of genomic and pathological data is often constrained by cost or technical limitations. To address this, we propose a novel Genomics-guided Multimodal Knowledge Decomposition Network (GMKDN), a framework that effectively integrates genomics and pathology data during training while dynamically adapting to available data during inference. GMKDN introduces two key innovations: (1) the Batch-Sample Multimodal Knowledge Decomposition (BMKD) module, which decomposes input features into pathology-specific, modality-general, and genomics-specific components to reduce redundancy and enhance knowledge transferability, and (2) the Online Similarity-Preserving Knowledge Distillation (OSKD) module, which optimizes activation similarity matrices to facilitate robust knowledge transfer between teacher and student models. The BMKD module improves generalization across modalities, while the OSKD module enhances model robustness, particularly when certain modalities are unavailable during inference. Extensive evaluations conducted on the TCGA-BRCA dataset and an external test cohort (QHSU) demonstrate that GMKDN consistently outperforms state-of-the-art (SOTA) slide-based multiple instance learning (MIL) approaches as well as existing multimodal learning models, establishing a new benchmark for breast cancer biomarker prediction. Our code is available at https://github.com/qiyuanzz/GMKDN.

Authors

Institutions

Publication Details

Journal
IEEE Transactions on Medical Imaging
Published
2025-01-01
DOI
https://doi.org/10.1109/tmi.2025.3628252
Primary Topic
AI in cancer detection
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Online Teaching: Distilling Decomposed Multimodal Knowledge for Breast Cancer Biomarker Prediction

Peng Gao, Yanmei Zhu, Q. Zhang, Cheng Lu et al.
IEEE Transactions on Medical Imaging
AI in cancer detection
article

Online Teaching: Distilling Decomposed Multimodal Knowledge for Breast Cancer Biomarker Prediction

Peng Gao, Yanmei Zhu, Q. Zhang, Cheng Lu, Fengyu Cong, Yaqi Du, Tong Wang, Hongming Xu, Xinyu Hao
article en

Abstract

Immunohistochemical (IHC) biomarker prediction greatly benefits from multimodal data fusion. However, the simultaneous acquisition of genomic and pathological data is often constrained by cost or technical limitations. To address this, we propose a novel Genomics-guided Multimodal Knowledge Decomposition Network (GMKDN), a framework that effectively integrates genomics and pathology data during training while dynamically adapting to available data during inference. GMKDN introduces two key innovations: (1) the Batch-Sample Multimodal Knowledge Decomposition (BMKD) module, which decomposes input features into pathology-specific, modality-general, and genomics-specific components to reduce redundancy and enhance knowledge transferability, and (2) the Online Similarity-Preserving Knowledge Distillation (OSKD) module, which optimizes activation similarity matrices to facilitate robust knowledge transfer between teacher and student models. The BMKD module improves generalization across modalities, while the OSKD module enhances model robustness, particularly when certain modalities are unavailable during inference. Extensive evaluations conducted on the TCGA-BRCA dataset and an external test cohort (QHSU) demonstrate that GMKDN consistently outperforms state-of-the-art (SOTA) slide-based multiple instance learning (MIL) approaches as well as existing multimodal learning models, establishing a new benchmark for breast cancer biomarker prediction. Our code is available at https://github.com/qiyuanzz/GMKDN.

IEEE Transactions on Medical ImagingVol. PP
Dalian University of Technology (CN), Dalian University (CN), First Hospital of China Medical University (CN), Qilu Hospital of Shandong University (CN), Southern Medical University (CN), China Medical University (CN)
Openalex Percentile: Top 83%
AI in cancer detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.