User Experience With an Automated Assessments Platform in an Introductory Programming Course

The teaching and learning in programming courses are often perceived as challenging for both students and teachers due to their complex and abstract nature. At the University of Tartu, hundreds of students with diverse backgrounds participate in introductory programming courses every semester. We use the platform Lahendus, which allows students to submit solutions and receive automated feedback on their programs. This study aims to investigate students' user experience in an introductory programming course. The sample included 64 persons (23.44 % males, 76.56 % females). The students differed in their background (i.e., curriculum, course type, experience in programming). The average age was 22.53 (19-49). We used System Usability Scale (SUS), Usability Metric for User Experience (UMUX), and User Experience Questionnaire (UEQ) to evaluate the user experience with Lahendus. Also, we aimed to find out the differences in user experience ratings of participants from different backgrounds. The survey consisted of 47 questions and statements and was divided into four topic blocks: background data, SUS, UMUX, and UEQ. The data were analyzed using quantitative methods. The results, in general, pointed to very good usability. The results also revealed that the user experience does not differ significantly based on gender. The study indicated that user experience ratings can be affected by the course being elective or compulsory in the curriculum. Overall, the study provided valuable insight into the user experience with the automated assessment system Lahendus in an introductory programming course and gave useful input for the platform development.