LAMP sheds light on how sequence to function models learn variant effects during post-training

Genomic sequence-to-function (S2F) models show promise for interpreting noncoding genetic variation, yet they currently underestimate the effects of most causal variants. This leads to fine-mapping failures that are reflected in their underperformance relative to linear baselines at predicting gene expression across individuals from personal genome sequences. Improving personal genome predictions, which reflect this underestimation, is therefore critical. There is broad interest in post-training S2F models to improve these predictions, yet many groups have shown that doing so with personal genome sequences and paired expression values does not meaningfully improve predictions on held-out genes and people (HOGP). Why these post-training efforts to improve predictions on HOGP, and therefore causal variant underestimation, remains unclear. To understand why, we introduce LAMP (Learning of Artificial Motif Perturbations), a framework in which artificial SNVs (aSNVs) perturbing transcription factor motifs are substituted into personal genome datasets genome-wide with large, consistent, and causal effects on gene expression. These aSNVs are designed to be deliberately easy to learn, yet fine-tuning Enformer using this LAMP framework led to persistent underperformance on HOGP, which suggests fundamental challenges for learning variant effects during post-training that are independent of the complexity of natural genetic variation. Our findings suggest this underperformance is caused in part due to challenges learning to strengthen enhancer-gene links during post-training. In particular, models struggle with learning to rewire attention weights to relate aSNV detection, over distance, to a change in expression of the target gene. This limitation is caused by strong pre-trained priors that downweight distal regulatory elements, causing extreme data inefficiency in learning to attend to distal elements, and variants within them, during post-training. Second, sequence variability between loci containing different instances of the same TF motif obscures otherwise easy-to-learn motif perturbation patterns and also hinder learning of variant effects. Our results suggest that the scale of observed human genetic variation is vastly insufficient to maximize S2F model accuracy on HOGP via simple fine-tuning, consistent with past outcomes. We propose data augmentation strategies that, under LAMP, significantly improve learning of aSNV effects by strategically targeting the limitations we identified. We release LAMP as a tool to troubleshoot and optimize future S2F post-training objectives.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-17
DOI
https://doi.org/10.64898/2026.11.13.766567
Primary Topic
Genomics and Chromatin Dynamics
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

LAMP sheds light on how sequence to function models learn variant effects during post-training

Katherine S. Pollard, Shiron Drusinsky
2 links
bioRxiv (Cold Spring Harbor Laboratory)
Genomics and Chromatin Dynamics
preprint

LAMP sheds light on how sequence to function models learn variant effects during post-training

Katherine S. Pollard, Shiron Drusinsky
preprint en

Abstract

Genomic sequence-to-function (S2F) models show promise for interpreting noncoding genetic variation, yet they currently underestimate the effects of most causal variants. This leads to fine-mapping failures that are reflected in their underperformance relative to linear baselines at predicting gene expression across individuals from personal genome sequences. Improving personal genome predictions, which reflect this underestimation, is therefore critical. There is broad interest in post-training S2F models to improve these predictions, yet many groups have shown that doing so with personal genome sequences and paired expression values does not meaningfully improve predictions on held-out genes and people (HOGP). Why these post-training efforts to improve predictions on HOGP, and therefore causal variant underestimation, remains unclear. To understand why, we introduce LAMP (Learning of Artificial Motif Perturbations), a framework in which artificial SNVs (aSNVs) perturbing transcription factor motifs are substituted into personal genome datasets genome-wide with large, consistent, and causal effects on gene expression. These aSNVs are designed to be deliberately easy to learn, yet fine-tuning Enformer using this LAMP framework led to persistent underperformance on HOGP, which suggests fundamental challenges for learning variant effects during post-training that are independent of the complexity of natural genetic variation. Our findings suggest this underperformance is caused in part due to challenges learning to strengthen enhancer-gene links during post-training. In particular, models struggle with learning to rewire attention weights to relate aSNV detection, over distance, to a change in expression of the target gene. This limitation is caused by strong pre-trained priors that downweight distal regulatory elements, causing extreme data inefficiency in learning to attend to distal elements, and variants within them, during post-training. Second, sequence variability between loci containing different instances of the same TF motif obscures otherwise easy-to-learn motif perturbation patterns and also hinder learning of variant effects. Our results suggest that the scale of observed human genetic variation is vastly insufficient to maximize S2F model accuracy on HOGP via simple fine-tuning, consistent with past outcomes. We propose data augmentation strategies that, under LAMP, significantly improve learning of aSNV effects by strategically targeting the limitations we identified. We release LAMP as a tool to troubleshoot and optimize future S2F post-training objectives.

bioRxiv (Cold Spring Harbor Laboratory)
Gladstone Institutes (US), University of California, San Francisco (US)
National Science Foundation, W. M. Keck Foundation, Gladstone Institutes
Good health and well-being
Genomics and Chromatin Dynamics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon