Engagements with GPT responses and learner prompts in ChatGPT-based learning of English argumentative writing logic and their impacts

ChatGPT can be defined as a chatbot powered by OpenAI’s GPT language models, which has shown promise in improving English-as-a-foreign-language (EFL) writing knowledge and skills. However, its application to developing EFL argumentative writing logic remains largely unexplored, despite the importance of this area. Moreover, existing studies have highlighted deep learner engagement with ChatGPT-based learning but have not examined how engagement varies between two key components of this learning method: GPT responses (GPT’s messages to learners) and learner prompts (learners’ messages to GPT). To better understand the mechanisms and efficacy of ChatGPT-based learning for EFL argumentative writing, we developed a discipline-specific GPT-4-powered chatbot for learning English argumentative writing logic. Forty-two Chinese university students used the tool for 45 to 75 minutes. Learner engagement in GPT responses and learner prompts was assessed via eye movements on corresponding interface areas of ChatGPT recorded by a Tobii eye-tracker. Their learning outcomes were assessed via pre-post-delayed tests and pre-post writing tasks. Semi-structured interviews were also administered. Our findings revealed that learners engaged with GPT responses frequently but for short durations, and with learner prompts infrequently but for longer durations. Engagement in GPT responses appears to facilitate logic knowledge development, whereas engagement in learner prompts may be associated with challenges in developing writing logic. Based on the results, we explored the factors influencing the patterns and impacts of learner engagement with ChatGPT-based learning of English argumentative writing logic and offered implications for future implementation of this learning method. • Learners engaged frequently but briefly with GPT responses; prompts less often, longer • Deep engagement with GPT responses enhanced argumentative writing logic knowledge • Engagement with learner prompts negatively affected writing logic development