Evaluator Reliability Under Selection Pressure: Implications for Agentic Test-Time Search
This paper investigates the reliability of learned evaluators under increasing Best-of-N selection pressure using fixed candidate pools generated by three language models across GSM8K and HumanEval. The study evaluates how selected-response correctness changes across increasing selection budgets while independently measuring task correctness. It identifies evaluator-dependent differences in selection reliability, including cases where increasing evaluator-score margins coincide with declining task accuracy. On GSM8K, DeepSeek PRM-selected accuracy for Llama 3.1 8B Instruct peaks at 80.4% at N = 8 before declining to 72.2% at N = 256, despite increasing evaluator-score margins. On HumanEval, CodeScaler demonstrates significant advantages over Skywork-Reward for Llama 3.1 8B Instruct and Mistral 7B Instruct, although this advantage is not established for Qwen3 14B after multiple-comparison correction. The findings motivate independent correctness audits across multiple selection budgets when evaluating learned evaluators for use in test-time search and agentic systems. Autonomous agents are not directly evaluated in this study. Workshop acceptance: Accepted for poster presentation at AIAS+ 2026: The 3rd Workshop on Lifelong Agents: Learning, Aligning, and Evolving (LLA). The author was unable to participate in the in-person poster session. Research conducted independently by the author.
Authors
- L Pradeep (ORCID: https://orcid.org/0009-0009-4150-1331)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-12
- DOI
- https://doi.org/10.5281/zenodo.24155959
- Primary Topic
- Evolutionary Algorithms and Applications
- Type
- preprint