Learnable Backdoors: Gotta Catch 'Em All
Can a backdoor poison be found in a pretraining corpus before any model is trained on it? Our blind two-stage screen shortlists strings that repeat across documents and precede a change of subject, then scores what follows them by a reference model’s excess surprise. With a cut-off from 12 fresh clean corpora alone, it flagged all nine poisoned 100,000-document corpora it was predicted to flag in a held-out set, fluent German and English payloads included, and none of nine held-out clean corpora; an attack built to escape it did. Its false-alarm rate, averaged over calibrations, is at most 1/(C+1) with C clean calibration corpora exchangeable with a clean test corpus, and, given how often its shortlist misses the trigger, its detection rate has an explicit lower bound. It rests on an identity: a poison’s gain to a trained model equals the payload’s total excess surprise plus two escape terms, the reference’s advantage over the clean model and its over-uncertainty, less a remainder that shrinks as the backdoor is learned. The identity names how a learned backdoor can hide, not that it cannot: at 63.5M parameters one escape term exceeds the gain, so there its detections of gibberish rest on the reference’s surprise, not the gain. At 12B tokens the screen failed and flagged the clean twin; post hoc, text that websites repeat had crowded the trigger out. We also show that AUROC does not determine a filter’s clean cost. Predictions were written before each test; we report the failures.
Authors
- Srihari Srinivasan (ORCID: https://orcid.org/0009-0009-2948-2449)
Institutions
- University of Warwick (GB)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-14
- DOI
- https://doi.org/10.5281/zenodo.24226660
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- preprint