Context Selection Has Been a Solved Problem Since 1951
An expository companion to the paper "Context Selection as Approximate Sufficiency," written in plain language and without the paper's machinery. Every large language model has a context window, and the window is always too small; something must be left out, and the choosing is the whole game. The field solves this daily under many names — RAG, context engineering, long-context memory, prompt compression — and none of them cite the mathematician who settled the question in 1951. The essay makes the case that context selection is a sufficiency problem. David Blackwell's comparison of experiments (1951) gives the partial order over compression methods: every compressor is a garbling of the stream, and Blackwell's theorem says exactly when one garbling dominates another. Lucien Le Cam's deficiency (1964) gives the targeted measure of how much a selected context loses for the one decision that matters — emulating the frozen model's output. Attention is shown to be the statistic; prompt compression, the garbling. The objects are the same objects; only the vocabulary changes. This is the outer half of a two-part foundation, the inner half being the structure-function essay. It is meant to be readable by anyone who works with transformers, ahead of the formal paper.
Authors
- Pia Alpila (ORCID: https://orcid.org/0009-0000-1457-7714)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-06-25
- DOI
- https://doi.org/10.5281/zenodo.20800030
- Primary Topic
- Machine Learning and Algorithms
- Type
- article
- Field-Weighted Citation Impact
- 0.00