LearnSQL: Impact of an Automatic Judge in Database Learning

Databases are a key topic in many technical university degrees. As databases have a strong practical nature, students are expected to solve many exercises before mastering the different aspects involved: querying and modifying the database, writing procedural code (functions and procedures), and defining triggers, among others. In this scenario, it is very important to have a substantial number of exercises available but also a timely feedback to detect and fix mistakes. Therefore, automatic judges that execute students’ solutions and generate immediate feedback are valuable tools to include in the teaching practice. In this article, we assess the real impact of using an online automatic judge for free practice in a database course over four academic years. For this purpose, we have contrasted the marks obtained in one academic year, without the automatic judge, against the three following years in which the automatic judge was used. The results show that final marks are statistically higher during the years when students make use of the automatic judge, thus showing an overall positive impact on database learning. Similarly, the results show that the more students use the automatic judge, the higher their final marks are. Besides these two insights, we have also studied if the impact of the automatic judge is the same in groups of high-profile students, concluding that this tool is less effective when improving learning in top-performing, highly self-motivated students in a database course.