Development and Application of TOPIK Speech Evaluation Feedback Model Based on ChatGPT

Thepurpose of this study is to design a TOPIK speaking evaluation feedback modeland prompt based on ChatGPT, a generative AI, and to verify its effectivenessthrough the 7th educational experiment and pre- and post-evaluation results. As averification method, the test group compared and analyzed the TOPIK speaking preandpost-evaluation scores of 11 Chinese learners in the experimental group towhich the feedback model was applied and 11 in the control group to which themodel was not applied. As a result of the analysis, the average difference in thepre- and post-evaluation scores of the experimental group was 18.31 points, andthe improvement effect of the experimental group was remarkably significantcompared to the control group’s 9.73 points. Significant differences were also foundin the post-evaluation response sample t-test results for the difference in the degreeof improvement between the two groups. In addition, in the three evaluation factors:‘content and task performance’, ‘language use’, and ‘speech delivery’, theexperimental group recorded a greater improvement of 5.8 points, 3.4 points, and6.8 points, respectively. These results suggest that the experimental group clearlyimproved the learning effect compared to the control group. Additionally, theparticipants demonstrated the model’s effectiveness once again in the experimentalgroup’s post-interview that the feedback model developed in this study clearlyimproved the learning effectiveness of the TOPIK speaking evaluation, and theparticipants of the control group also showed positive expectations for aChatGPT-based feedback model. This study demonstrated that the use of feedbackfunctions from Generative AI tools such as ChatGPT, focusing on TOPIK speakingevaluation, can actually help Korean learners improve their TOPIK speakinglearning ability. It is also significant in that it clearly demonstrates the potential of Generative AI-based learning tools such as ChatGPT in TOPIK speakinglearning.