Robust angle-based transfer learning in high dimensions

Transfer learning improves target model performance by leveraging data from related source populations, especially when target data are scarce. This study addresses the challenge of training high-dimensional regression models with limited target data in the presence of heterogeneous source populations. We focus on a practical setting where only parameter estimates of pretrained source models are available, rather than individual-level source data. For a single source model, we propose a novel angle-based transfer learning (angleTL) method that leverages concordance between source and target model parameters. AngleTL adapts to the signal strength of the target model, unifies several benchmark methods, and mitigates negative transfer when between-population heterogeneity is large. We extend angleTL to incorporate multiple source models, accounting for varying levels of relevance among them. Our high-dimensional asymptotic analysis provides insights into when a source model benefits the target model and demonstrates the superiority of angleTL over other methods. Extensive simulations validate these findings and highlight the feasibility of applying angleTL to transfer genetic risk prediction models across multiple biobanks.

Authors

Institutions

Publication Details

Journal
Journal of the Royal Statistical Society Series B (Statistical Methodology)
Published
2024-12-03
DOI
https://doi.org/10.1093/jrsssb/qkae111
Citations
8
Primary Topic
Cancer-related molecular mechanisms research
Type
article
Field-Weighted Citation Impact
2.28

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Robust angle-based transfer learning in high dimensions

Tian Gu, Rui Duan, Yi Han
8 citations
Journal of the Royal Statistical Society Series B (Statistical Methodology)
Cancer-related molecular mechanisms research
2.28
article

Robust angle-based transfer learning in high dimensions

Tian Gu, Rui Duan, Yi Han
article en
8 citations

Abstract

Transfer learning improves target model performance by leveraging data from related source populations, especially when target data are scarce. This study addresses the challenge of training high-dimensional regression models with limited target data in the presence of heterogeneous source populations. We focus on a practical setting where only parameter estimates of pretrained source models are available, rather than individual-level source data. For a single source model, we propose a novel angle-based transfer learning (angleTL) method that leverages concordance between source and target model parameters. AngleTL adapts to the signal strength of the target model, unifies several benchmark methods, and mitigates negative transfer when between-population heterogeneity is large. We extend angleTL to incorporate multiple source models, accounting for varying levels of relevance among them. Our high-dimensional asymptotic analysis provides insights into when a source model benefits the target model and demonstrates the superiority of angleTL over other methods. Extensive simulations validate these findings and highlight the feasibility of applying angleTL to transfer genetic risk prediction models across multiple biobanks.

Journal of the Royal Statistical Society Series B (Statistical Methodology)Vol. 87(3)
Cancer Research And Biostatistics (US), Columbia University (US)
National Institutes of Health
Openalex Percentile: Top 10%
Cancer-related molecular mechanisms research
2.28
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.