Identifying Barriers to Using Generative AI in Education Through AI-Assisted Literature Mining of arXiv Articles

This study presents an exploratory, generative AI (GenAI)-assisted full-text analysis of barriers to using GenAI in education within an arXiv corpus. A total of 19,756 records were retrieved using a keyword search and then screened for relevance using three large language models (LLMs), GPT-4o mini, Gemini 2.0 flash-lite, and Llama3-70B (Groq), through a voting mechanism. This yielded a corpus of 1998 education-relevant full-text articles. Using GPT-4o, learning and teaching scenarios (LTSs) were extracted using a structured 6As framework (Actor, Activity, AI Role, AI Interface, Assessment, Achievement) developed for this study. Barrier themes were derived separately by clustering a total of 4729 barrier descriptions, embedded using OpenAI’s embedding model. The descriptive results indicate: (i) STEM (science, technology, engineering, and mathematics) vs. non-STEM distributions of GenAI use cases are broadly comparable; (ii) approximately 75% of reported GenAI intervention outcomes were classified as non-negative across the different disciplines; and (iii) reported barriers show different descriptive patterns between STEM and non-STEM disciplines. STEM use cases more frequently report performance constraints, while non-STEM ones more frequently report privacy and workflow integration concerns. The observed disciplinary differences in extracted barriers suggest that approaches to supporting GenAI use may need to vary by disciplinary context. However, the findings are limited by arXiv coverage and limited human validation; future research should broaden source coverage and strengthen robust validation.