Goodhart's Law in the Classroom and the Algorithm: Why Static Benchmarks Fail Both Human Learners and Artificial Intelligence

AbstractStandardised assessments and artificial intelligence benchmarks can produce high performance scores without necessarily demonstrating an equivalent ability to apply knowledge under changed conditions. This conceptual working paper examines that measurement problem through a comparison of secondary-school examinations and large language model mathematical reasoning benchmarks.Drawing on CBSE Circular Acad-30/2024, the GSM-Symbolic research presented at ICLR 2025, and scholarship on performance measurement, the paper considers how familiarity with predictable question structures may complicate the interpretation of assessment results. It distinguishes human learning from machine evaluation while identifying a shared concern: whether a measured score adequately captures performance when relevant assumptions or problem conditions change.The paper proposes three practical assessment modifications: controlled re-parameterisation, assumption reversal, and explanation-based error diagnosis. It also outlines a potential classroom evaluation framework addressing transfer, assessment validity, accessibility, and marking reliability.The proposed modifications are intended for further investigation rather than presented as experimentally validated improvements.Publication status: Independent, non-peer-reviewed conceptual working paper (November 2026). No original classroom experiments, participant studies, or model benchmark tests were conducted. AI-assisted drafting and editorial support are disclosed in the manuscript.

Authors

Publication Details

Journal
Figshare
Published
2026-11-16
DOI
https://doi.org/10.6084/m9.figshare.35016610
Primary Topic
Educational Assessment and Pedagogy
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Goodhart's Law in the Classroom and the Algorithm: Why Static Benchmarks Fail Both Human Learners and Artificial Intelligence

ahmed younus laskar
2 links
Figshare
Educational Assessment and Pedagogy
preprint

Goodhart's Law in the Classroom and the Algorithm: Why Static Benchmarks Fail Both Human Learners and Artificial Intelligence

ahmed younus laskar
preprint en

Abstract

AbstractStandardised assessments and artificial intelligence benchmarks can produce high performance scores without necessarily demonstrating an equivalent ability to apply knowledge under changed conditions. This conceptual working paper examines that measurement problem through a comparison of secondary-school examinations and large language model mathematical reasoning benchmarks.Drawing on CBSE Circular Acad-30/2024, the GSM-Symbolic research presented at ICLR 2025, and scholarship on performance measurement, the paper considers how familiarity with predictable question structures may complicate the interpretation of assessment results. It distinguishes human learning from machine evaluation while identifying a shared concern: whether a measured score adequately captures performance when relevant assumptions or problem conditions change.The paper proposes three practical assessment modifications: controlled re-parameterisation, assumption reversal, and explanation-based error diagnosis. It also outlines a potential classroom evaluation framework addressing transfer, assessment validity, accessibility, and marking reliability.The proposed modifications are intended for further investigation rather than presented as experimentally validated improvements.Publication status: Independent, non-peer-reviewed conceptual working paper (November 2026). No original classroom experiments, participant studies, or model benchmark tests were conducted. AI-assisted drafting and editorial support are disclosed in the manuscript.

Figshare
Quality education
Educational Assessment and Pedagogy
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon