Short versus Short: The Long-Horizon Baselines in "You Don't Need Long Context" Were Mostly Short

Belcak et al. (2026) argue that long contexts are unnecessary; their own long-horizon tables do not establish that conclusion. On the five long-horizon benchmarks that report history lengths, the paper's uncompacted baselines have median per-task histories of 7.2K–32K tokens, against the paper's own reference of approximately 256K tokens for "long context". Its headline economic advantage depends on a hypothetical context-dependent pricing scenario: at the nominal API prices reported in the same paper, compaction lowered GPT 5.6 Sol's aggregate token use in 17 of 22 configurations yet raised its nominal cost in 21 of 22. Its task-accuracy comparisons are single-run and reported without uncertainty; using the reported rounded accuracies and stated task counts, a conditional bound under an exact two-sided McNemar test on binary task outcomes shows that at least 26 of the 41 non-zero long-horizon differences could not reach significance at the 0.05 level under any consistent paired outcome pattern. The long-input experiments show that selective retrieval enables short-context execution on the evaluated tasks, but they change input access and context length together and so do not isolate the effect of context length. These findings do not invalidate short-context engineering. They challenge a more consequential inference: that successful execution under short-context conditions establishes that long context is unnecessary. Preprint, not peer reviewed. Version 1.0.