AI-Enhanced Quality of Multimodal Physics Word Problems in Authentic Contexts Using the P-MAGIC App

This study aimed to develop an app called Physics Multimodal Authentic Generated Interactive Content Problem (P-MAGIC) to gather authentic contextual information by using object recognition and real-time sensors to help students learn the principles of centripetal acceleration in Physics. We implemented an instructional-based prompt generator that adjusts the difficulty level of problems in three levels, with the questions themselves being generated by the Generative Pre-trained version 4o (GPT-4o) model. Furthermore, to evaluate our app, we assessed the quality of the generated questions using automatic and human evaluations. The automatic evaluation demonstrated that the questions were relevant, easy to understand, and diverse. Additionally, feedback from teachers showed high ratings for relevance, completeness, and concept accuracy. Interestingly, problems incorporating both text and images, followed by text, images, and graphs, outperformed those that were based on text and graphs or text and tables. This highlights the positive impact of visual elements on conceptual understanding and student engagement. Furthermore, the strong correlation between automatic and human evaluations indicates that automated assessments can reliably reflect human judgement, making them a scalable tool for evaluations. By providing teachers with the tools to create personalized and context-rich problems in a variety of formats could potentially transform physics education.