Understanding the Impact of Using Generative AI Tools in a Database Course

Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) have led to changes in educational practices by creating opportunities for personalized learning and immediate support. Computer science student perceptions and behaviors towards GenAI tools have been studied, but the effects of such tools on student learning have yet to be determined conclusively. We investigate the impact of GenAI tools on computing students' performance in a database course and aim to understand why students use GenAI tools in assignments. Our mixed-methods study (N=226) asked students to self-report whether they used a GenAI tool to complete a part of an assignment and why. Our results reveal that students utilizing GenAI tools performed better on the assignment part in which LLMs were permitted but did worse in other parts of the assignment and in the course overall. Also, those who did not use GenAI tools viewed more discussion board posts and participated more than those who used ChatGPT. This suggests that using GenAI tools may not lead to better skill development or mental models, at least not if the use of such tools is unsupervised, and that engagement with official course help supports may be affected. Further, our thematic analysis of reasons for using or not using GenAI tools, helps understand why students are drawn to these tools. Shedding light into such aspects empowers instructors to be proactive in how to encourage, supervise, and handle the use or integration of GenAI into courses, fostering good learning habits.