The Frame Ladder: How the phrasing of a request changes reward-hacking rates
Working note reading Figures 12 and 13 of Qi et al., Training a Misaligned Reward Seeker (Anthropic Alignment Science, August 2026), with secondary reference to OpenAI's Hugging Face incident report. All figures quoted are the original authors'; the readings and the labels are mine. Reports the effect of four prompt framings on reward-hacking rates in an impossible-task evaluation, and two open measurement questions arising from it.
Authors
- Yanush Feshter (ORCID: https://orcid.org/0009-0002-1330-7530)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-01
- DOI
- https://doi.org/10.5281/zenodo.22236693
- Primary Topic
- Personal Information Management and User Behavior
- Type
- preprint