Context Selection Has Been a Solved Problem Since 1951

An expository companion to the paper "Context Selection as Approximate Sufficiency," written in plain language and without the paper's machinery. Every large language model has a context window, and the window is always too small; something must be left out, and the choosing is the whole game. The field solves this daily under many names — RAG, context engineering, long-context memory, prompt compression — and none of them cite the mathematician who settled the question in 1951. The essay makes the case that context selection is a sufficiency problem. David Blackwell's comparison of experiments (1951) gives the partial order over compression methods: every compressor is a garbling of the stream, and Blackwell's theorem says exactly when one garbling dominates another. Lucien Le Cam's deficiency (1964) gives the targeted measure of how much a selected context loses for the one decision that matters — emulating the frozen model's output. Attention is shown to be the statistic; prompt compression, the garbling. The objects are the same objects; only the vocabulary changes. This is the outer half of a two-part foundation, the inner half being the structure-function essay. It is meant to be readable by anyone who works with transformers, ahead of the formal paper.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-25
DOI
https://doi.org/10.5281/zenodo.20800030
Primary Topic
Machine Learning and Algorithms
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Context Selection Has Been a Solved Problem Since 1951

Pia Alpila
Zenodo (CERN European Organization for Nuclear Research)
Machine Learning and Algorithms
article

Context Selection Has Been a Solved Problem Since 1951

Pia Alpila
article en

Abstract

An expository companion to the paper "Context Selection as Approximate Sufficiency," written in plain language and without the paper's machinery. Every large language model has a context window, and the window is always too small; something must be left out, and the choosing is the whole game. The field solves this daily under many names — RAG, context engineering, long-context memory, prompt compression — and none of them cite the mathematician who settled the question in 1951. The essay makes the case that context selection is a sufficiency problem. David Blackwell's comparison of experiments (1951) gives the partial order over compression methods: every compressor is a garbling of the stream, and Blackwell's theorem says exactly when one garbling dominates another. Lucien Le Cam's deficiency (1964) gives the targeted measure of how much a selected context loses for the one decision that matters — emulating the frozen model's output. Attention is shown to be the statistic; prompt compression, the garbling. The objects are the same objects; only the vocabulary changes. This is the outer half of a two-part foundation, the inner half being the structure-function essay. It is meant to be readable by anyone who works with transformers, ahead of the formal paper.

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Openalex Percentile: Top 5%
Machine Learning and Algorithms
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.