Evaluating large language models in high school physics education: addressing misconceptions and fostering conceptual understanding

Abstract Large language models (LLMs), which are a specific subset of artificial intelligence (AI), may have the potential to revolutionize education by addressing common student misconceptions in physics. This study investigates the effectiveness of popular LLMs, such as OpenAI ChatGPT, Google Gemini, and Microsoft Copilot, in identifying and addressing misconceptions related to Newtonian mechanics in high school physics. The focus solely is on the misconception that for an elevator to move upward, the force exerted by the cable must be greater than the gravitational force acting downward on the elevator, which contradicts Newton’s first law of motion. To explore this, thirty-two experienced instructors engaged in dialogues with the LLMs, simulating learners with this misconception. Instructors evaluated the LLMs’ accuracy, personalization, and pedagogical effectiveness. The findings indicate that most instructors recognized the substantial potential of LLMs to improve student learning, particularly in addressing misconceptions through interactive dialogue, targeted questioning, and clear explanations tailored to each learner’s needs. ChatGPT ranked highest, demonstrating capabilities in delivering clear explanations, adapting to individual learners, and implementing effective teaching strategies. Google Gemini followed closely, while Microsoft Copilot was the least effective. This capability holds promise for enhancing conceptual understanding and student engagement in physics education. However, limitations were noted in the LLMs’ ability to facilitate personalized scientific discussions and utilize visual aids, such as physics diagrams, simulations, to enhance understanding. This research demonstrates the significant potential of LLMs as valuable tools for identifying and addressing misconceptions in physics education.