Cross-modality Knowledge Transfer Between fMRI and EEG for Improved Single-modality Alzheimer’s Disease Diagnosis

Alzheimer's disease (AD) disrupts brain function in both spatial organization and temporal dynamics, which can be characterized by complementary fMRI and EEG measurements. In particular, fMRI provides higher spatial resolution to characterize multi-scale functional organization of brain networks, whereas EEG offers higher temporal resolution to capture dynamic neural activities across frequency bands. Thus, integrating the two modalities can characterize AD-related spatiotemporal disruptions more comprehensively than either modality alone. However, it is impractical to acquire paired fMRI and EEG data in real-world applications. To address this problem, we propose the fMRI-EEG Knowledge Transfer Network (FE-KTNet) to learn spatiotemporal representations from paired data and then transfer them back to modality-specific models through three stages. Specifically, in Stage 1, it learns modality-specific representations from fMRI and EEG FCNs, emphasizing multi-scale network organization for fMRI and frequency-specific electrophysiological coupling for EEG. Building on these representations, in Stage 2, it performs scale-wise bidirectional cross-modality fusion to integrate complementary information into fused multi-scale spatiotemporal representations. Finally, in Stage 3, it transfers the fused representations back to each modality-specific encoder through scale-wise feature alignment and logit-level knowledge distillation. We evaluate FE-KTNet on 14 multicenter datasets comprising 2,766 fMRI subjects and 1,842 EEG subjects, covering AD-NC, MCI-NC, and AD-MCI classifications. Compared with the corresponding baseline models, FE-KTNet increases fMRI-only and EEG-only accuracy by 3.4% and 6.1% for AD-NC classification, 5.2% and 8.0% for MCI-NC classification, and 3.5% and 7.1% for AD-MCI classification, respectively, and achieves the best overall performance among all evaluated methods. Our code is available at GitHub Repository.

Authors

Institutions

Publication Details

Journal
IEEE Transactions on Medical Imaging
Published
2026-11-18
DOI
https://doi.org/10.1109/tmi.2026.3754829
Primary Topic
Dementia and Cognitive Impairment Research
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Cross-modality Knowledge Transfer Between fMRI and EEG for Improved Single-modality Alzheimer’s Disease Diagnosis

Dinggang G. Shen, Jiaxing Xu, Feng Shi, 郭艳苏 et al.
IEEE Transactions on Medical Imaging
Dementia and Cognitive Impairment Research
article

Cross-modality Knowledge Transfer Between fMRI and EEG for Improved Single-modality Alzheimer’s Disease Diagnosis

Dinggang G. Shen, Jiaxing Xu, Feng Shi, 郭艳苏, Ye He, Zhiming Cui, Wu Zhao, Xinyan Du, Minhui Tan, Yulong Dou, Yuxiao Liu, Weiqi Li, Xin Lin, Wenfei Yu, Jingjing Lin, Jiafan Qin
article en

Abstract

Alzheimer's disease (AD) disrupts brain function in both spatial organization and temporal dynamics, which can be characterized by complementary fMRI and EEG measurements. In particular, fMRI provides higher spatial resolution to characterize multi-scale functional organization of brain networks, whereas EEG offers higher temporal resolution to capture dynamic neural activities across frequency bands. Thus, integrating the two modalities can characterize AD-related spatiotemporal disruptions more comprehensively than either modality alone. However, it is impractical to acquire paired fMRI and EEG data in real-world applications. To address this problem, we propose the fMRI-EEG Knowledge Transfer Network (FE-KTNet) to learn spatiotemporal representations from paired data and then transfer them back to modality-specific models through three stages. Specifically, in Stage 1, it learns modality-specific representations from fMRI and EEG FCNs, emphasizing multi-scale network organization for fMRI and frequency-specific electrophysiological coupling for EEG. Building on these representations, in Stage 2, it performs scale-wise bidirectional cross-modality fusion to integrate complementary information into fused multi-scale spatiotemporal representations. Finally, in Stage 3, it transfers the fused representations back to each modality-specific encoder through scale-wise feature alignment and logit-level knowledge distillation. We evaluate FE-KTNet on 14 multicenter datasets comprising 2,766 fMRI subjects and 1,842 EEG subjects, covering AD-NC, MCI-NC, and AD-MCI classifications. Compared with the corresponding baseline models, FE-KTNet increases fMRI-only and EEG-only accuracy by 3.4% and 6.1% for AD-NC classification, 5.2% and 8.0% for MCI-NC classification, and 3.5% and 7.1% for AD-MCI classification, respectively, and achieves the best overall performance among all evaluated methods. Our code is available at GitHub Repository.

IEEE Transactions on Medical ImagingVol. PP
Beijing University of Posts and Telecommunications (CN), National University of Singapore (SG), Capital Medical University (CN), ShanghaiTech University (CN), United Imaging Intelligence (China) (CN)
National Natural Science Foundation of China
Good health and well-being
Dementia and Cognitive Impairment Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon