A Comparative Study of the Effectiveness of Feedback from ChatGPT or Native Speakers of English on Low English Proficient Adult Learners.

This study explored the effects of feedback provided by the generative AI ChatGPT versus native speakers (NS) on the improvement of English grammar skills and feedback perception among adult learners with low English proficiency (TOEIC scores below 450). Thirty-nine graduate students (19 in the experimental ChatGPT group, 20 in the control native speaker group) from a university in D Metropolitan City participated in a 6-week feedback activity, with data from 27 participants analyzed. Google Sheets was used to share feedback, and non-parametric statistical analyses (Mann-Whitney U test, Wilcoxon Signed Rank test) were employed. The analysis revealed no statistically significant difference in English grammar proficiency between the two groups in either pre- or post-tests. Although both groups showed improved mean scores on the post-test, the extent of improvement was not statistically significant. Analysis of feedback perception through surveys indicated that the ChatGPT group positively evaluated the reduction in English writing anxiety and immediate feedback, but expressed concerns about feedback accuracy and lack of explanation. Conversely, the native speaker feedback group expressed satisfaction with grammar correction, learning opportunities, and insights into sentence structure, likening the experience to one-on-one mentoring. In conclusion, while quantitative analysis showed no significant difference between the two feedback types regarding grammar improvement, learners perceived both methods as meaningful activities. ChatGPT had strengths in reducing anxiety and providing immediacy, while native speaker feedback was highly valued for its clarity and grammatical instruction.