Integrating AI chatbots in informal digital English learning: impacts on listening competencies in Chinese higher education

With the rapid advancement of technology, an increasing number of learners are turning to AI chatbots in Informal Digital Learning of English (IDLE) to enhance their language skills. However, empirical research on leveraging this emerging mode of learning for listening development remains limited. This quasi-experimental mixed-methods study explores the integration of an AI chatbot (ChatGPT-4) in IDLE and its impact on the listening competencies of Chinese tertiary-level EFL students. 67 students participated in the quantitative phase (experimental class [EC] = 35; control class [CC] = 32), with pre- and post-tests conducted over a 16-week intervention. Qualitative data were collected through semi-structured interviews with 10 EC students after the intervention. During the study, EC students engaged in CET-4-format listening tasks via ChatGPT-4 and received individualized feedback in an IDLE context. Quantitative results demonstrated significant gains in listening proficiency among EC participants, with large to very large within-group effect sizes (Cohen’s d = 1.57–3.43) and substantial between-group differences (Cohen’s d = 1.05–2.26). Thematic analysis of interview data revealed that students found the chatbot’s interactive and adaptive features conducive to a more engaging and personalized learning experience. This research addresses a gap in understanding the role of AI technology in IDLE contexts, highlighting the pedagogical potential of AI chatbots to enhance listening skills and enrich out-of-class learning environments. The study offers important implications for language educators and curriculum designers seeking to foster learner autonomy and enhance listening skills through emerging technologies.