Teaching a language model to read DNA: fine-tuning Muse Glimmer 30B on cell-free DNA methylation with a vision-style adapter

Large language models read text, code and images, but not biological measurements. We ask whether an open-weight language model can learn to read DNA methylation measured on cell-free DNA (cfDNA) from a blood sample, and answer two questions in plain words: is there a tumour signal, and which tissue does it come from? We adapt Meta's open-weight Muse Glimmer 30B with the recipe that taught language models to see: a small adapter projects 3,148 per-CpG methylation features into 16 soft tokens in the model's embedding space, and the model is fine-tuned with QLoRA (217M trained parameters, 66 minutes on one L40S GPU). The answer is read from the model's own next-token logits. We document all six training runs carried out in one week, including failures (mode collapse, memorisation of blood donors) and the changes that fixed them (two-stage training, pseudo-donors, cross-cohort batch alignment). Trained on 35,000 plasma samples simulated from public data (3,832 tumours from 22 series, 5,728 blood donors from 8 cohorts, 23 tissue classes) and evaluated on people and tumours never seen in training, the final model finds 75% of signals at 1% tumour DNA with 99.5% specificity, and names the right tissue first 58% of the time among 23 classes, against 46% for a gradient-boosting baseline on the same data. These results are on simulated plasma, with a small sanity check on 27 real plasma samples where the language model's alarm does not yet transfer well; the work is a research prototype, not a diagnostic. Code, weights, run reports and a live demo are public: https://github.com/zoukmar/burotu · https://huggingface.co/zoukmar75/burotu-glimmer · https://burotu.com

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-04
DOI
https://doi.org/10.5281/zenodo.23941917
Primary Topic
Cancer Genomics and Diagnostics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Teaching a language model to read DNA: fine-tuning Muse Glimmer 30B on cell-free DNA methylation with a vision-style adapter

Michael Marzouk
Zenodo (CERN European Organization for Nuclear Research)
Cancer Genomics and Diagnostics
preprint

Teaching a language model to read DNA: fine-tuning Muse Glimmer 30B on cell-free DNA methylation with a vision-style adapter

Michael Marzouk
preprint en

Abstract

Large language models read text, code and images, but not biological measurements. We ask whether an open-weight language model can learn to read DNA methylation measured on cell-free DNA (cfDNA) from a blood sample, and answer two questions in plain words: is there a tumour signal, and which tissue does it come from? We adapt Meta's open-weight Muse Glimmer 30B with the recipe that taught language models to see: a small adapter projects 3,148 per-CpG methylation features into 16 soft tokens in the model's embedding space, and the model is fine-tuned with QLoRA (217M trained parameters, 66 minutes on one L40S GPU). The answer is read from the model's own next-token logits. We document all six training runs carried out in one week, including failures (mode collapse, memorisation of blood donors) and the changes that fixed them (two-stage training, pseudo-donors, cross-cohort batch alignment). Trained on 35,000 plasma samples simulated from public data (3,832 tumours from 22 series, 5,728 blood donors from 8 cohorts, 23 tissue classes) and evaluated on people and tumours never seen in training, the final model finds 75% of signals at 1% tumour DNA with 99.5% specificity, and names the right tissue first 58% of the time among 23 classes, against 46% for a gradient-boosting baseline on the same data. These results are on simulated plasma, with a small sanity check on 27 real plasma samples where the language model's alarm does not yet transfer well; the work is a research prototype, not a diagnostic. Code, weights, run reports and a live demo are public: https://github.com/zoukmar/burotu · https://huggingface.co/zoukmar75/burotu-glimmer · https://burotu.com

Zenodo (CERN European Organization for Nuclear Research)
Cancer Genomics and Diagnostics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon