Bugbear or surefire success? Secondary school students’ conceptual learning about evolution with ChatGPT

Evolution is challenging to understand for students. When engaging with challenging topics, students increasingly use the support of AI tools such as ChatGPT. Nevertheless, it is largely unclear how interacting with ChatGPT affects students’ learning success. Thus, we conducted an experimental intervention study (<i>N</i> = 241 upper secondary school students) with two intervention groups (i.e. control and experimental group) and a pre- post-follow-up-test design to investigate the effects of using ChatGPT on conceptual knowledge about evolution, digital competence regarding AI-supported learning, and attitude toward AI-supported learning. The control group attempted to expand their evolutionary knowledge through traditional internet searches, while the experimental group used ChatGPT for learning. Results show correlations between general, evolution-related, and AI-related variables. Furthermore, findings indicate that ChatGPT provides a limited enhancement regarding students’ conceptual knowledge. Although AI-supported learning had no significant effects on digital competence, students in the experimental group developed a more positive attitude toward AI-supported learning. An increased positive attitude despite a very limited enhancement of learning may pose risks, as students may use ChatGPT frequently for future learning despite its limited effectiveness. Researchers and educators need to find ways to enable students to engage with AI in a reflective way.