Analyzing the Impact of Formal Assessment Modalities: Comparative Study in Data Science for Engineering Education

This study examines the impact of changes in exam modalities on the performance and experiences of architectural engineering students in a domain-specific data science class. Specifically, the number and duration of exams (and thereby the amount of content on each) and setting in which the students took the exams in changed among the three years examined. The analysis involved comparing student grades, sentiment analysis of written feedback, and keyword analysis between the Spring semesters of 2021, 2022, and 2023. The results demonstrate significant improvements in overall grades and mean exam grades in 2022 and 2023 compared to 2021, indicating that the shift in offering more frequent and shorter in-class exams covering less material positively impacted student performance. This indication is also supported by the keyword analysis, which revealed a shift in student feedback discussing challenges with formal assessments. Furthermore, sentiment analysis shows less polarization and a significant decrease in negative comments after compared to before the intervention, suggesting improved student perception and experience in the course. By considering both quantitative and qualitative measures, this study provides a comprehensive understanding of how student performance and experiences changed with different exam modalities, guiding future curriculum design and instructional improvements for data science courses for engineers.