Performance Is Not Learning: Designing courses for the age of AI, and four principles from the evidence
Generative AI has made one of the oldest distinctions in learning science newly urgent: performance during practice is not the same as learning that lasts. In a randomized field experiment, students practicing with a ChatGPT-style assistant scored 48% higher during practice and 17% lower once the assistant was removed. In a randomized trial at Harvard, a structured AI tutor more than doubled median learning gains over an in-class active-learning lesson. The technology was comparable; the design was not. This article reads those trials alongside course-level studies of optional guardrails, oral code reviews, and the link between confidence and critical thinking. It proposes four design principles for courses where AI is present, considers what they mean for higher education institutions, and states the limits of the evidence.
Authors
- Clark Ngo (ORCID: https://orcid.org/0009-0004-2951-4790)
Institutions
- City University of Seattle (US)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-13
- DOI
- https://doi.org/10.5281/zenodo.24204020
- Primary Topic
- Artificial Intelligence in Education
- Type
- preprint