Comparing live and video observations in preschool classrooms: Evidence from within-observation CLASS scores

This study provides evidence on the comparability of live and video classroom observation scores. Observations were conducted from May 2023 to August 2024 in 59 preschool classrooms serving children ages 3-5 across five U.S. states. Among classroom lead teachers, 87% identified as women; 6% identified as Asian, 27% as Black, 18% as Hispanic or Latine, 36% as White, and 5% as multiracial. Using the Classroom Assessment Scoring System, observers conducted 101 live observations while simultaneously recording videos to be coded by a different observer. Within-observation fixed-effects methods were used to compare live and video scores for the same observation. Video scores for Emotional Support were lower compared to live scores (d = -0.30). Additionally, whether live and video scores differed was moderated by observation scores and language. Specifically, video scores across all domains were lower than live scores in higher quality observations, but video scores for Classroom Organization and Instructional Support were higher than live scores in lower quality observations. Video scores were higher than live scores for Instructional Support in multilingual Spanish-English observations, but there were no differences in monolingual English observations. The findings have implications for using video observations for early care and education research, coaching, and accountability.