Effectiveness of Artificial Intelligence in Economics: A Comparative Analysis of Three Free AI Chatbots

This study aims to explore how AI-powered Chatbots can transform economics education by delivering real-time information, analyses, and insights into economic concepts. We conduct a comparative analysis of three prominent free AI Chatbots (i.e., Perplexity AI, Claude 3 Sonnet, and Meta Llama 3 70B) to evaluate the quality and comprehensiveness of each Chatbot's knowledge base to appraise their effectiveness in responding to nuanced economic inquiries across Macroeconomics and Microeconomics domains. Using the tool Test of Understanding of College Economics (TUCE) version 4 questionnaire with 30 questions per domain, this study assesses the accuracy and comprehensiveness of these Chatbots’ responses using qualitative content and comparative score analyses. This study leads us to vouch for AI chatbots as being acceptable as guides in the case of economic theories and problem-solving. Another finding is that the observation of variance in the performance of AI Chatbots is due to chance, revoking the idea of difference in the depth of true knowledge, in case of economics. By identifying strengths and limitations, this research provides valuable insights for educators integrating advanced technologies into teaching and supports students in leveraging interactive AI-driven platforms to enhance their understanding of economics.