Using AI chatbots to improve university EFL students’ speech scripts' writing quality and critical thinking skills

The purpose of this study was to investigate effects of artificial intelligence (AI) chatbots on university English as a foreign language (EFL) students’ language use and critical thinking skills in English speech script writing. The 18-week English-speaking course was designed and conducted on the basis of production-oriented approach, a theory of foreign language education with Chinese features. By integrating the production-oriented approach with the technological capabilities of the chatbot, the course instructor offered 30 university EFL students theoretical instruction and task-driven writing activities focused on public speaking in English. Throughout the course, the participants worked in collaboration with the AI chatbot to complete each script writing assignment. Ultimately, the instructor collected and analyzed all original drafts, revised drafts, and transcripts of conversations between the students and the AI chatbot. The study’s results showed that: (1) four indicators of lexical complexity in the revised drafts were significantly improved compared with the original drafts (p < 0.01), whereas one indicator of syntactic complexity also demonstrated significant enhancement (p < 0.05); (2) in comparison with the original drafts, the majority of the revised drafts (71.93%) showed modifications in regard to claims, warrants, and grounds; (3) the contents of the conversations between students and AI chatbot were grouped into three themes: making suggestions for language revisions, extending and developing the main points, and providing supporting materials for arguments. The study’s results can be taken as a reference by English teachers for designing and conducting writing and speaking courses. The study also contributes empirical evidence towards building AI-assisted EFL teaching theories and methods in the era of AI.