An investigation of using Spark generative AI in solving physics concept inventories in English and Chinese: performance and issues

Generative artificial intelligence (GenAI) has garnered considerable attention across various disciplines, including physics education. Numerous studies have explored the potential of using these tools in physics education by assessing their understanding of physics concepts. However, ChatGPT is the only model whose performance and integration into physics education have been extensively studied. Furthermore, previous research has primarily focused on English as the input language, leaving a gap in our understanding of other models and languages. This study aims to address this gap by examining the performance of Spark, another GenAI developed in China, in solving physics concept inventories. Four conditions were investigated: English input without explanation, English input with explanation, Chinese input without explanation, and Chinese input with explanation. The results showed that Spark’s performance with English input was comparable to ChatGPT3.5 for the Force Concept Inventory but significantly lagged behind ChatGPT4. Notably, Chinese input with explanation significantly outperformed the other three conditions. This study also discussed concerns and issues related to Spark’s physics conceptual understanding and language inequality. Finally, guidelines for incorporating GenAI into physics education were proposed.