Agreement and Bias in ChatGPT-Assisted Grading of Undergraduate Nursing Examinations: A Method-Comparison Study

BACKGROUND: Agreement between ChatGPT-assisted and faculty grading of nursing examinations remains uncertain. PURPOSE: To compare ChatGPT-assisted and faculty grading across item types and examine bias, repeatability, pass-fail classification, and major scoring discrepancies. METHODS: We analyzed 506 undergraduate nursing examinations using paired comparisons, intraclass correlation coefficients, error measures, and Bland-Altman analysis. Multiple-choice repeatability and 150 constructed-response scoring units with major overestimations were also examined. RESULTS: Multiple-choice grading showed minimal bias (-0.05 points) and excellent agreement with faculty grading (intraclass correlation coefficient = 0.949). ChatGPT overestimated the constructed-response subtotal and total scores by 7.48 and 7.43 points. Of the 59 examinations with faculty total scores below the institutional passing threshold of 60 points, ChatGPT classified 40 as passing. Major overestimations mainly reflected insufficient deductions and over-crediting of generalized or rubric-external content. CONCLUSIONS: ChatGPT showed high agreement for fixed-key multiple-choice scoring but systematically overestimated constructed-response scores. Faculty review remains necessary for consequential assessment decisions.