Criterion-Purpose Drift Under Optimization: How a Measure Can Keep Improving as Its Link to Its Purpose Weakens

This conceptual paper introduces Criterion-Purpose Drift Under Optimization, a mechanism in which a criterion remains in use while its relationship to its intended purpose weakens. Continued effort directed at the criterion can preserve the mismatch even as scores improve. The paper assesses drift through evidence about the criterion’s relationship to its purpose, including independent measurements or examination of how scores are produced. A rising score or a poor initial indicator does not by itself establish that the relationship has weakened. Drift can arise through honest adaptation or changing conditions and need not involve deception. The discussion connects the mechanism to Campbell’s work on indicators used in decisions and uses RAND’s summary of a 1998 Kentucky assessment study. The study’s authors judged gains on the state assessment substantially inflated relative to comparison tests. The inference depends on the comparability of the tests, and coaching’s contribution was not quantified. Part of the Standards and Answerability series within On Human Modeling, the paper asks who can reassess and revise an operational standard. Its framework can inform human–AI research on whether benchmark gains continue to track intended capabilities and user outcomes, and whether continued optimization leaves a weakened relationship uncorrected.