Preregistration: Do AI chatbots ground their grading of student essays in the text? A feasibility pilot

Preregistration of a feasibility pilot that tests how three AI chatbots (ChatGPT, Gemini, Grok) grade three German student essays from the GENIUS Keylog Corpus: whether their quotes occur in the essay, how stable the grades are over three repetitions, and whether they detect one inserted error per essay. 54 grading runs, done by hand. Registered before any run. The study state is frozen at commit bfcd872 of github.com/LAKITALKS/Inspektorium. The Markdown file is identical to studies/school-pilot-2026/prereg/preregistration.md (SHA-256 760dbbb3593f76bbfcf13d2d45891830a1212b6371823f0fa27ee40752c864d0).

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-11
DOI
https://doi.org/10.5281/zenodo.24131847
Primary Topic
Artificial Intelligence in Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Preregistration: Do AI chatbots ground their grading of student essays in the text? A feasibility pilot

Lazaros Varvatis
Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Education
article

Preregistration: Do AI chatbots ground their grading of student essays in the text? A feasibility pilot

Lazaros Varvatis
article en

Abstract

Preregistration of a feasibility pilot that tests how three AI chatbots (ChatGPT, Gemini, Grok) grade three German student essays from the GENIUS Keylog Corpus: whether their quotes occur in the essay, how stable the grades are over three repetitions, and whether they detect one inserted error per essay. 54 grading runs, done by hand. Registered before any run. The study state is frozen at commit bfcd872 of github.com/LAKITALKS/Inspektorium. The Markdown file is identical to studies/school-pilot-2026/prereg/preregistration.md (SHA-256 760dbbb3593f76bbfcf13d2d45891830a1212b6371823f0fa27ee40752c864d0).

Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Education
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Preregistration: Do AI chatbots ground their grading of student essays in the text? A feasibility pilot — Lazaros Varvatis · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS