A Study on the use of AI Chatbot feedback in argumentative writing education and learner acceptance patterns － Focusing on the development of GPTs-based customized feedback Chatbots and analysis of educational effects

In this study, we designed and operated an argumentative writing feedback AI chatbot based on ChatGPT and analyzed learners’ perceptions in detail in order to explore practical ways to apply the core competencies of AI literacy - information seeking, critical analysis, and problem solving - to university writing education. The study was conducted in three stages: first, we set the evaluation criteria to evaluate learners’ writing based on the evaluation criteria of ‘content, organization, and expression’; second, we developed an AI chatbot that evaluates learners’ reports based on the evaluation criteria and provides specific directions for revision and supplementation. Third, we asked participants to write two reports using the AI chatbot and investigated their perceptions of the AI chatbot’s feedback. The 50 participants in the experiment wrote a draft report and a revised version that incorporated the AI chatbot feedback, and three experts analyzed the changes in the quality of the report according to the evaluation criteria. Participants who used the AI chatbot for report writing perceived that the chatbot feedback was more systematic and objective than traditional face-to-face feedback, and that it was more effective in providing direction for revision, especially in improving sentence expression, strengthening logical connections, and clarifying evidence. In addition, the detailed evaluation criteria provided by the chatbot promoted learners' metacognition and self-regulated learning, and the iterative questioning and feedback process made a significant positive contribution to strengthening learners’ ability to recognize weaknesses in their writing and set direction for revision. This study empirically confirmed that an AI chatbot based on ChatGPT-4o can function as a meaningful feedback tool in argumentative writing education, and suggested that chatbot feedback is effective in improving learners’ self-directed writing skills and can be complementary to face-to-face feedback.