Learning with AI: A Critical Comparative Study of Chatbots’ Potential in Upper Secondary School Mathematics

The use of generative AI in schools is becoming increasingly common among both teachers and students. However, as it is a new technology, there is a risk that the level of knowledge (AI-literacy) among the stakeholders is not sufficient to meet the challenges as well as fully benefit from the advantages that it can provide. This study focuses on three of the most popular services at the time of writing: ChatGPT, Gemini and Copilot. The objective is to contribute to the improvement of AI literacy and its potential application in high-school mathematics teaching. The material used consists of three different integral related tests conducted on the generative AIs, as well as interviews with high-school teachers who critically evaluate those results in a Swedish school context. A mixed method approach is used, combining hermeneutics and positivism. Hermeneutics is a tool that allows the interpretation of the material while positivism complements the hermeneutic approach by providing a more objective insight. According to the teachers’ interpretation of the studied material, although the three chatbots are capable of correctly solving most of the high-school level mathematical problems and integrals, they still have a relatively high number of mistakes (hallucinations), and this can make their unsupervised use by the students misleading.