PROGMF: Progressive Multimodal Fusion and Representation Learning for Incomplete Neuroimaging Data

Multimodal neuroimaging studies often involve cohorts with partially overlapping data modalities. Many conventional multimodal fusion methods rely on participants with complete observations across the selected modalities. We introduce Progressive Multimodal Fusion (ProgMF), a novel fusion framework for learning multimodal source representations from incomplete neuroimaging data without imputing missing observations. ProgMF integrates modalities sequentially through independent vector analysis (IVA), cross-stage component alignment, and anchor-guided representation transfer, allowing information learned from earlier-stage cohorts to inform subsequent multimodal decompositions. We evaluated ProgMF on four neuroimaging-inspired synthetic modalities, while varying subject overlap, observation noise, modality relatedness, and fusion orders. ProgMF generally achieved lower source-recovery errors than complete-case IVA and the evaluated imputation-based methods, with larger improvements as multimodal subject overlap decreased. We further applied ProgMF to structural MRI gray matter, resting-state functional network connectivity, and 18F-florbetapir (AV45) amyloid positron emission tomography data from the Alzheimer's Disease Neuroimaging Initiative. ProgMF achieved higher mean cross-modal fusion scores than the imputation-based baselines and a modest improvement over complete-case IVA. Among the 293 participants with complete multimodal observations, two recovered source component vectors showed significant associations with clinical diagnosis across cognitively normal, mild cognitive impairment, and dementia groups. Both sources were also significantly associated with cognitive performance and dementia severity. The recovered sources exhibited diagnosis-related structural and amyloid PET patterns involving regions such as the hippocampi, precuneus, and cingulate cortices. Our findings support progressive representation transfer as an effective approach for recovering coherent, interpretable multimodal sources from participants with incomplete modality coverage.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-18
DOI
https://doi.org/10.64898/2026.11.17.767205
Primary Topic
Functional Brain Connectivity Studies
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

PROGMF: Progressive Multimodal Fusion and Representation Learning for Incomplete Neuroimaging Data

Rafal A. Angryk, Rogers F. Silva, Vince D. Calhoun, Tülay Adalı et al.
bioRxiv (Cold Spring Harbor Laboratory)
Functional Brain Connectivity Studies
preprint

PROGMF: Progressive Multimodal Fusion and Representation Learning for Incomplete Neuroimaging Data

Rafal A. Angryk, Rogers F. Silva, Vince D. Calhoun, Tülay Adalı, Jingyu Liu, Victor Solomon
preprint en

Abstract

Multimodal neuroimaging studies often involve cohorts with partially overlapping data modalities. Many conventional multimodal fusion methods rely on participants with complete observations across the selected modalities. We introduce Progressive Multimodal Fusion (ProgMF), a novel fusion framework for learning multimodal source representations from incomplete neuroimaging data without imputing missing observations. ProgMF integrates modalities sequentially through independent vector analysis (IVA), cross-stage component alignment, and anchor-guided representation transfer, allowing information learned from earlier-stage cohorts to inform subsequent multimodal decompositions. We evaluated ProgMF on four neuroimaging-inspired synthetic modalities, while varying subject overlap, observation noise, modality relatedness, and fusion orders. ProgMF generally achieved lower source-recovery errors than complete-case IVA and the evaluated imputation-based methods, with larger improvements as multimodal subject overlap decreased. We further applied ProgMF to structural MRI gray matter, resting-state functional network connectivity, and 18F-florbetapir (AV45) amyloid positron emission tomography data from the Alzheimer's Disease Neuroimaging Initiative. ProgMF achieved higher mean cross-modal fusion scores than the imputation-based baselines and a modest improvement over complete-case IVA. Among the 293 participants with complete multimodal observations, two recovered source component vectors showed significant associations with clinical diagnosis across cognitively normal, mild cognitive impairment, and dementia groups. Both sources were also significantly associated with cognitive performance and dementia severity. The recovered sources exhibited diagnosis-related structural and amyloid PET patterns involving regions such as the hippocampi, precuneus, and cingulate cortices. Our findings support progressive representation transfer as an effective approach for recovering coherent, interpretable multimodal sources from participants with incomplete modality coverage.

bioRxiv (Cold Spring Harbor Laboratory)
Georgia Institute of Technology (US), Emory University (US), Georgia State University (US), Center for Translational Research in Neuroimaging and Data Science (US), University of Maryland, Baltimore County (US)
National Science Foundation, National Institutes of Health
Good health and well-being
Functional Brain Connectivity Studies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon