The Frame Ladder: How the phrasing of a request changes reward-hacking rates

Working note reading Figures 12 and 13 of Qi et al., Training a Misaligned Reward Seeker (Anthropic Alignment Science, August 2026), with secondary reference to OpenAI's Hugging Face incident report. All figures quoted are the original authors'; the readings and the labels are mine. Reports the effect of four prompt framings on reward-hacking rates in an impossible-task evaluation, and two open measurement questions arising from it.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-01
DOI
https://doi.org/10.5281/zenodo.22236693
Primary Topic
Personal Information Management and User Behavior
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Frame Ladder: How the phrasing of a request changes reward-hacking rates

Yanush Feshter
Zenodo (CERN European Organization for Nuclear Research)
Personal Information Management and User Behavior
preprint

The Frame Ladder: How the phrasing of a request changes reward-hacking rates

Yanush Feshter
preprint en

Abstract

Working note reading Figures 12 and 13 of Qi et al., Training a Misaligned Reward Seeker (Anthropic Alignment Science, August 2026), with secondary reference to OpenAI's Hugging Face incident report. All figures quoted are the original authors'; the readings and the labels are mine. Reports the effect of four prompt framings on reward-hacking rates in an impossible-task evaluation, and two open measurement questions arising from it.

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Personal Information Management and User Behavior
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.