ChatGPT as Economics Tutor: Capabilities and Limitations

Since the public release of ChatGPT in late 2022, the role of Generative AI chatbots in education has been widely debated. While some see their potential as automated tutors, others worry that inaccuracies and hallucinations could harm student learning. Thisstudy assesses ChatGPT models (GPT-3.5, GPT-4o, and o1preview) across important dimensions of student learning by evaluating their capabilities and limitations to serve as a non-interactive, automated tutor. In particular, we analyse performance in two tasks commonly used in principles of economics courses: explaining economic concepts and answering multiple-choice questions. Our findings indicate that newer models generate very accurate responses, although some inaccuracies persist. A key concern is that ChatGPT presents all responses with full confidence, making errors difficult for students to recognize. Furthermore, explanations are often quite narrow, lacking holistic perspectives, and the quality of examples remains poor. Despite these limitations, we argue that ChatGPT can serve as an effective automated tutor for basic, knowledge-based questions—supportingstudents while posing a relatively low risk of misinformation. Educators can hence recommend Generative AI chatbots for student learning, but should teach students the limitations of the technology.