Comparing ChatGPT, Peer, and Expert Assessments of Pre-Service Primary Teachers’ Language Teaching Competence

Generative AI is increasingly used in educational assessment, but its contribution to evaluating complex teaching competence remains uncertain. This comparative thematic study examined assessment evidence generated by ChatGPT Edu, uncalibrated peer evaluators, and one expert evaluator for 27 pre-service primary teachers in a single university language-teaching methodology course. The corpus comprised 135 reports: 27 ChatGPT Edu, 81 peer, and 27 expert assessments. AI-assisted thematic analysis of the full corpus was complemented by an independent human audit of nine cases (45 reports). Three themes emerged: From Presence to Proof, Active Is Not Enough, and Agreement Is Layered. Overall evaluative judgements sometimes converged, but their evidential justifications, diagnostic specificity, and assessment objects differed. Across evaluator perspectives, recurrent patterns concerned the evidential thresholds separating the identification of pedagogical features from claims about their enactment and consequences, and learner activity from claims about communicative agency, engagement, and inclusion. Because ChatGPT Edu’s processing of acoustic information could not be verified, judgements requiring acoustic evidence were excluded from cross-evaluator comparison. The findings suggest that different assessment perspectives may offer complementary evidence, but do not establish that combining them improves validity or teacher learning. Responsible pedagogical use of ChatGPT-generated feedback therefore requires attention to evidential justification, evidence accessibility, and human professional judgement.

Authors

Institutions

Publication Details

Journal
Education Sciences
Published
2026-11-17
DOI
https://doi.org/10.3390/educsci16111972
Primary Topic
Student Assessment and Feedback
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Comparing ChatGPT, Peer, and Expert Assessments of Pre-Service Primary Teachers’ Language Teaching Competence

Lenka Rovňanová, Dana Hanesová
3 links
Education Sciences
Student Assessment and Feedback
article

Comparing ChatGPT, Peer, and Expert Assessments of Pre-Service Primary Teachers’ Language Teaching Competence

Lenka Rovňanová, Dana Hanesová
article en

Abstract

Generative AI is increasingly used in educational assessment, but its contribution to evaluating complex teaching competence remains uncertain. This comparative thematic study examined assessment evidence generated by ChatGPT Edu, uncalibrated peer evaluators, and one expert evaluator for 27 pre-service primary teachers in a single university language-teaching methodology course. The corpus comprised 135 reports: 27 ChatGPT Edu, 81 peer, and 27 expert assessments. AI-assisted thematic analysis of the full corpus was complemented by an independent human audit of nine cases (45 reports). Three themes emerged: From Presence to Proof, Active Is Not Enough, and Agreement Is Layered. Overall evaluative judgements sometimes converged, but their evidential justifications, diagnostic specificity, and assessment objects differed. Across evaluator perspectives, recurrent patterns concerned the evidential thresholds separating the identification of pedagogical features from claims about their enactment and consequences, and learner activity from claims about communicative agency, engagement, and inclusion. Because ChatGPT Edu’s processing of acoustic information could not be verified, judgements requiring acoustic evidence were excluded from cross-evaluator comparison. The findings suggest that different assessment perspectives may offer complementary evidence, but do not establish that combining them improves validity or teacher learning. Responsible pedagogical use of ChatGPT-generated feedback therefore requires attention to evidential justification, evidence accessibility, and human professional judgement.

Education SciencesVol. 16(11)
Matej Bel University (SK)
Quality education
Student Assessment and Feedback
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon