A comparative evaluation of Large Language Models for efficient AI feedback design: Evidence from undergraduate persuasive writing tasks

This study compares the feedback profiles produced by ChatGPT and Gemini on the same undergraduate persuasive-writing task under an identical prompt. Using an RTPR-based LLM-as-a-Judge framework with counter-balancing, we tested seven rubric criteria and the overall mean via independent two-sample t-tests. Significant differences emerged for most criteria and for the overall mean (p＜.001); Lexical Accuracy favored ChatGPT over Gemini (p=.017), whereas Sentence Clarity was not significant (p=.084). These findings indicate systematic, model-specific strengths and weaknesses even under identical inputs, suggesting instructionally targeted model choice (e.g., argument/evidence/organization vs. lexical refinement). The study was limited to two models and a single genre; no multiple-comparison correction was applied.