Examining the Effectiveness of Generative AI as a Bilingual Dictionary in Korean EFL Contexts

Large Language Model-based generative AI has the potential to change language education, as it is able to parse natural language prompts and provide natural-seeming responses to these prompts. To date, there has been little research into generative AI as a tool to assist language learners in reading comprehension and vocabulary acquisition. This paper presents an experimental study that compares the effectiveness of ChatGPT to the online bilingual NAVER Dictionary and to a control group in both a reading comprehension/written recall task and a vocabulary acquisition test using both the Vocabulary Knowledge Scale test and a multiple choice meaning test. The subjects were 120 Korean undergraduate students majoring in English Language and Literature. ANOVA results of pausal units and target vocabulary recalled during the written recall task suggest that a dedicated bilingual dictionary is superior to ChatGPT for reading comprehension, but AI can assist learners to identify unknown words just as well. For vocabulary acquisition, ANOVA results on both vocabulary measures show both ChatGPT and NAVER Dictionary groups significantly outperformed the control group, with no differences between the two dictionary groups. This suggests that ChatGPT may be an adequate substitute for a bilingual dictionary and a useful tool for language learners to improve vocabulary.