Myths of official measurement: Limits to test-based education reforms with weak governance

Assessment-led school reforms are a central pillar of policy packages recommended to address low student achievement in developing countries. We use direct audit evidence to assess the truthfulness of official assessments in a reform that has tested over 6 million students annually since 2011 in the large Indian state of Madhya Pradesh. Comparing responses to the same test questions by the same students shows a doubling of reported achievement in administrative data versus independent tests. This difference is lower, within schools, in grades with multiple test booklets and external grading. Overall, in contexts with weak governance, interventions relying on test-based accountability appear unlikely to succeed without complementary investments to assure data integrity. • Standardized tests underlie many education reforms in developing countries. • Direct audits show that official tests substantially overstate learning in India. • Discrepancy between official and independent tests results reflects manipulation. • Test-based education reforms require complementary investments in data integrity.