Language models transmit behavioural traits through hidden signals in data

. Here we show that distillation can lead to subliminal learning-the transmission of behavioural traits through semantically unrelated data. In our main experiments, a 'teacher' model with some trait T (such as disproportionately generating responses favouring owls or showing broad misaligned behaviour) generates datasets consisting solely of number sequences. Remarkably, a 'student' model trained on these data learns T, even when references to T are rigorously removed. More realistically, we observe the same effect when the teacher generates math reasoning traces or code. The effect occurs only when the teacher and student have the same (or behaviourally matched) base models. To help explain this, we prove a theoretical result showing that subliminal learning arises in neural networks under broad conditions and demonstrate it in a simple multilayer perceptron (MLP) classifier. As artificial intelligence systems are increasingly trained on the outputs of one another, they may inherit properties not visible in the data. Safety evaluations may therefore need to examine not just behaviour, but the origins of models and training data and the processes used to create them.

Authors

Institutions

Publication Details

Journal
Nature
Published
2026-04-15
DOI
https://doi.org/10.1038/s41586-026-10319-8
Citations
2
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
56.68

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Language models transmit behavioural traits through hidden signals in data

Sören Mindermann, James Chua, Anna Sztyber, Jan Betley et al.
2 citations
Nature
Topic Modeling
56.68
article

Language models transmit behavioural traits through hidden signals in data

Sören Mindermann, James Chua, Anna Sztyber, Jan Betley, Samuel Marks, Owain Evans, Jacob Hilton, Minh Hoang Le, Alex Cloud
article en
2 citations

Abstract

. Here we show that distillation can lead to subliminal learning-the transmission of behavioural traits through semantically unrelated data. In our main experiments, a 'teacher' model with some trait T (such as disproportionately generating responses favouring owls or showing broad misaligned behaviour) generates datasets consisting solely of number sequences. Remarkably, a 'student' model trained on these data learns T, even when references to T are rigorously removed. More realistically, we observe the same effect when the teacher generates math reasoning traces or code. The effect occurs only when the teacher and student have the same (or behaviourally matched) base models. To help explain this, we prove a theoretical result showing that subliminal learning arises in neural networks under broad conditions and demonstrate it in a simple multilayer perceptron (MLP) classifier. As artificial intelligence systems are increasingly trained on the outputs of one another, they may inherit properties not visible in the data. Safety evaluations may therefore need to examine not just behaviour, but the origins of models and training data and the processes used to create them.

NatureVol. 652(8110)
Warsaw University of Technology (PL), American Legacy Foundation (US), Anthropological Survey of India (IN), American Anthropological Association (US), Institute for Anthropological Research (HR), Align Technology (United States) (US), Cambridge University Press (GB), Science Oxford (GB), Laboratoire d'Anthropologie Sociale (FR), University of California, Berkeley (US)
Open Philanthropy Project
Quality Education
Openalex Percentile: Top 0%
Topic Modeling
56.68
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.