Cross-modality Knowledge Transfer Between fMRI and EEG for Improved Single-modality Alzheimer’s Disease Diagnosis
Alzheimer's disease (AD) disrupts brain function in both spatial organization and temporal dynamics, which can be characterized by complementary fMRI and EEG measurements. In particular, fMRI provides higher spatial resolution to characterize multi-scale functional organization of brain networks, whereas EEG offers higher temporal resolution to capture dynamic neural activities across frequency bands. Thus, integrating the two modalities can characterize AD-related spatiotemporal disruptions more comprehensively than either modality alone. However, it is impractical to acquire paired fMRI and EEG data in real-world applications. To address this problem, we propose the fMRI-EEG Knowledge Transfer Network (FE-KTNet) to learn spatiotemporal representations from paired data and then transfer them back to modality-specific models through three stages. Specifically, in Stage 1, it learns modality-specific representations from fMRI and EEG FCNs, emphasizing multi-scale network organization for fMRI and frequency-specific electrophysiological coupling for EEG. Building on these representations, in Stage 2, it performs scale-wise bidirectional cross-modality fusion to integrate complementary information into fused multi-scale spatiotemporal representations. Finally, in Stage 3, it transfers the fused representations back to each modality-specific encoder through scale-wise feature alignment and logit-level knowledge distillation. We evaluate FE-KTNet on 14 multicenter datasets comprising 2,766 fMRI subjects and 1,842 EEG subjects, covering AD-NC, MCI-NC, and AD-MCI classifications. Compared with the corresponding baseline models, FE-KTNet increases fMRI-only and EEG-only accuracy by 3.4% and 6.1% for AD-NC classification, 5.2% and 8.0% for MCI-NC classification, and 3.5% and 7.1% for AD-MCI classification, respectively, and achieves the best overall performance among all evaluated methods. Our code is available at GitHub Repository.
Authors
- Dinggang G. Shen (ORCID: https://orcid.org/0000-0002-7934-5698)
- Jiaxing Xu (ORCID: https://orcid.org/0000-0003-2498-5812)
- Feng Shi (ORCID: https://orcid.org/0000-0003-1522-9943)
- 郭艳苏
- Ye He (ORCID: https://orcid.org/0000-0001-7113-8056)
- Zhiming Cui (ORCID: https://orcid.org/0000-0002-3798-4504)
- Wu Zhao (ORCID: https://orcid.org/0000-0001-5811-3911)
- Xinyan Du
- Minhui Tan (ORCID: https://orcid.org/0009-0000-4134-4609)
- Yulong Dou (ORCID: https://orcid.org/0009-0007-0558-9159)
- Yuxiao Liu (ORCID: https://orcid.org/0009-0003-8298-5420)
- Weiqi Li (ORCID: https://orcid.org/0009-0000-4188-2730)
- Xin Lin (ORCID: https://orcid.org/0009-0005-2360-4637)
- Wenfei Yu
- Jingjing Lin
- Jiafan Qin
Institutions
- Beijing University of Posts and Telecommunications (CN)
- National University of Singapore (SG)
- Capital Medical University (CN)
- ShanghaiTech University (CN)
- United Imaging Intelligence (China) (CN)
Publication Details
- Journal
- IEEE Transactions on Medical Imaging
- Published
- 2026-11-18
- DOI
- https://doi.org/10.1109/tmi.2026.3754829
- Primary Topic
- Dementia and Cognitive Impairment Research
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Natural Science Foundation of China