Do student perceptions mirror reality? A mixed-methods study to evaluate the accuracy, utility and efficiency of AI-assisted feedback in introductory mathematics courses

Abstract Background Giving meaningful feedback on assignments in large introductory-level mathematics courses is a challenge. The emergence of “out of the box” Large Language Models (e.g., ChatGPT, Gemini) has created a potential solution to this challenge, where the AI tool can be trained to give constructive feedback on student work. However, it is unclear whether these tools are sufficiently accurate, efficient, and trustworthy. The current study investigates the use of a bespoke AI tool to assist in giving feedback to students in two introductory mathematics courses that enroll over 400 students each and are characterized by high attrition. Results We evaluated the similarities in grades assigned by the AI tool and human graders, as well as student and grader perceptions of the feedback given by both the AI tool and human graders. We found that the tool was fairly accurate, but that perceptions of the utility of its feedback varied by condition. When evaluated as a stand-alone grading agent, the AI tool was significantly undervalued relative to human graders—both student peers and the professor. Conclusion Three themes that characterize our findings: student trust is key, the professor is irreplaceable, and graders and AI tools must be carefully trained to ensure accuracy, transparency, and trust. We conclude with a call to exercise caution in using these tools in grading assistance, particularly with respect to maintaining student trust and transparency in grading.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-14
DOI
https://doi.org/10.21203/rs.3.rs-11455953/v1
Primary Topic
Artificial Intelligence in Education
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Do student perceptions mirror reality? A mixed-methods study to evaluate the accuracy, utility and efficiency of AI-assisted feedback in introductory mathematics courses

Antonella Zanna Munthe-Kaas, Christian Bru, Therese Romslo Saltskår, Sehoya Cotner et al.
Research Square
Artificial Intelligence in Education
preprint

Do student perceptions mirror reality? A mixed-methods study to evaluate the accuracy, utility and efficiency of AI-assisted feedback in introductory mathematics courses

Antonella Zanna Munthe-Kaas, Christian Bru, Therese Romslo Saltskår, Sehoya Cotner, Kasper Troøyen
preprint en

Abstract

Abstract Background Giving meaningful feedback on assignments in large introductory-level mathematics courses is a challenge. The emergence of “out of the box” Large Language Models (e.g., ChatGPT, Gemini) has created a potential solution to this challenge, where the AI tool can be trained to give constructive feedback on student work. However, it is unclear whether these tools are sufficiently accurate, efficient, and trustworthy. The current study investigates the use of a bespoke AI tool to assist in giving feedback to students in two introductory mathematics courses that enroll over 400 students each and are characterized by high attrition. Results We evaluated the similarities in grades assigned by the AI tool and human graders, as well as student and grader perceptions of the feedback given by both the AI tool and human graders. We found that the tool was fairly accurate, but that perceptions of the utility of its feedback varied by condition. When evaluated as a stand-alone grading agent, the AI tool was significantly undervalued relative to human graders—both student peers and the professor. Conclusion Three themes that characterize our findings: student trust is key, the professor is irreplaceable, and graders and AI tools must be carefully trained to ensure accuracy, transparency, and trust. We conclude with a call to exercise caution in using these tools in grading assistance, particularly with respect to maintaining student trust and transparency in grading.

Research Square
University of Bergen (NO)
Artificial Intelligence in Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon