Association between on-demand video lectures and Common Achievement Tests scores in pre-clinical education

Abstract Background Since the COVID-19 pandemic, on-demand lectures have been widely adopted in university education. However, the association between on-demand lectures and learning has not been objectively and comprehensively investigated. This retrospective cohort study compared pre-pandemic and post-pandemic student cohorts and examined the association with performance on the Pre-Clinical Clerkship Common Achievement Tests. Methods A total of 643 test records from fourth-year medical students at a Japanese national university were used for this retrospective cohort study. The cohort before the implementation of on-demand video lectures was classified as the no-video lectures group, and the cohort after the implementation as the video lectures group. Learning outcomes were assessed using the Pre-Clinical Clerkship Common Achievement Tests, including the Computer Based Testing (CBT) and Objective Structured Clinical Examination (OSCE), and a generalized linear model (GLM) was used to examine group, grade point average (GPA), and their interaction. Results After applying the exclusion criteria, 629 records were analyzed. The video lectures group had a significantly higher GPA than the no-video lectures group. CBT scores were significantly higher in the video lectures group, although the effect size was small ( d = 0.411). No significant difference in OSCE scores was observed between the two groups. In the GLM, a significant interaction between GPA and group was observed, and the association between GPA and CBT performance was stronger in the cohort after the implementation of on-demand video lectures. Conclusions These findings suggest that the association between academic performance and knowledge assessment outcomes may differ between cohorts. However, this study was a retrospective cohort study, and GPA differed between the groups. Therefore, the observed associations cannot be attributed solely to on-demand video lectures. Further prospective studies with adequate control of confounding factors are warranted.