Generative AI in mathematics education: analysing student performance and perceptions over three academic years

This study presents an approach in which generative AI (GenAI) chatbots are used to assist students in a problem-solving task resulting in a mathematical essay, introduced in the academic year 2022/2023 in a higher education mathematics course, and continued with in 2023/2024. We compared students' results from these two years with the results from 2021/2022, when GenAI was not part of the task, using Kruskal-Wallis and Dunn tests. The total sample included 966 students. The 2022/2023 results (GenAI introduced) differed significantly from the other two years, with students being less successful. Conversely, 2023/2024 (GenAI standard) showed the best student results and no significant difference from 2021/2022 (no GenAI). In all the three years, results in the problem-solving essay correlated with success in previous formative and summative assessments. Student feedback highlighted advantages and disadvantages of interactions with GenAI, revealing a tendency to utilise GenAI more for theoretical than practical tasks.