What Does the First Level of a Residual Text Tokenizer Know? Learned-Query Pooling Puts the Gist in Level 1

Coarse-to-fine text generators such as Next-Concept Modeling (NCM) rely on a residual vector-quantized tokenizer whose first levels should carry the gist of a sentence. NCM builds each coarse level by pooling fixed windows of positions. We replace this with learned-query pooling (CruxTok, T1) and train on XSum summaries. Learned pooling makes level 1 recover much more of a sentence: against an NCM baseline with the same parameter count, T1's level-1 reconstructions reach LLM-judge win rates of 0.74 and 0.77 (two seeds, ties counted half; the judge picks T1 outright in 57% and 63% of items and the baseline in 9% and 8%), and embedding similarity to the original rises from 0.34 to 0.45. T1 also reconstructs 13–15 points more tokens from the first three levels. The pooling attention moves away from its starting prior and depends on the input. The parameter-matched baseline, which spends its extra parameters on encoder and decoder depth, gives codes whose similarity tracks STS-B better (Spearman 0.28 at level 3, against 0.02–0.03 for T1). Decodable meaning and meaning-organized codes are separate properties, and pooling buys the first but not the second. We also report the stabilizer needed to train learned pooling at all, QK-normalization against attention-logit growth. Code, configurations, evaluation outputs and trained tokenizers are public.