Exploring Authentic Assessment in the GenAI Era: The Efficacy of Student-Created Screencasts, an Exploration Using Explainable AI (XAI)

The proliferation of generative AI (GenAI) in higher education necessitates a shift towards more authentic assessment methods that can genuinely measure student understanding. Student-created screencasts are a potentially valuable yet underexamined assessment format in higher education. A screencast is a screen capture video, which records all the activities on a computer screen. This study examines the efficacy of student-created screencasts as an authentic assessment. Using data from 84 undergraduate students, Elastic Net regression and Shapley Additive Explanations (SHAP) analysis were applied to examine predictive relationships across assessment types: screencasts with audio narration (SWAs), screencasts without audio (SNAs), written assignments, and group project presentations. The findings reveal that SWAs have a moderate-to-strong positive association with test performance, acting as a significantly stronger predictor than both traditional written work and SNAs. The results suggest that narrated screencasts may serve as a useful assessment format for gauging student understanding, supporting the early identification of at-risk students, and providing a platform for developing crucial communication skills. This assessment strategy holds promise for enhancing academic integrity and learning in the GenAI era. This study contributes to assessment research by demonstrating the predictive value of narrated screencasts in identifying at-risk students in AI-supported learning environments. Future research should extend this work through multi-institutional designs, longitudinal analysis, and qualitative investigation of student perceptions in various courses other than artificial intelligence.