Beyond Alignment: The Retrieval Value of Offline Teacher Information

Abstract A compressed retriever can gain substantial retrieval quality from teacher information that its alignment examples never constrain. We isolate this resource by changing only teacher-token responses outside the support of 2,048 fitting queries. Source features, cross-validation and fitted coefficients remain identical, while retrieval improves. Across two encoder settings and three tasks, the full teacher table exceeds the champion of a separate 30-configuration source-only affine, kernel and neighbor search in each encoder setting by .05165 ArguAna, .01334 NFCorpus and .08793 Android nDCG@10. The selected nonlinear completions improve source-coordinate reconstruction yet lose target relevance relative to the original affine rule. Revealing more exact teacher rows can likewise lower retrieval quality along a fixed acquisition order. These interventions separate information accuracy from retrieval value: a response matters through the query–document decisions it changes after pooling and readout. Retained priors provide a complementary source-null witness, and matched-readout comparisons show substantial remaining value in deep full-query computation above the offline table. The resulting account of query compression connects inherited information and offline teacher acquisition to the relevance earned by online computation. It motivates selecting teacher information for its effect on retrieval decisions.