Unveiling the Impact of Generative AI on Language Learning: A Field Experiment

While the diffusion of generative AI in educational contexts has the potential to transform learning, empirical examinations of its effectiveness are lacking. Our field experiment compared learning outcomes of English speaking between participants engaging with generative AI and those interacting with human conversational agents. We discovered that generative AI significantly enhances the complexity of spoken content, with its impact on learning outcomes comparable to that of human agents. Notably, we observed that generative AI is particularly effective in promoting attentive learning among individuals with lower proficiency in English. We further examined the role of evaluation apprehension in driving these differential effects. Our findings suggest that generative AI can serve as an effective standalone learning tool and potentially mitigate disparities in educational background proficiency.