When student evaluation of teaching is no longer indicative: Psychology students’ in-class experience, motivational factors and future choices

Student evaluations of teaching (SETs) are widely used in educational institutions to assess instructional quality, but concerns about potential negative effects are growing. We developed a new SET instrument, administered multiple times throughout the semester, focused on aspects students could meaningfully evaluate and instructors could act upon in large lecture psychology classrooms. Factor analyses of data collected from 883 students across three different psychology courses, we identified five distinct factors. An Active Learning factor positively predicted grades. Belonging and Inclusion, Productive Questioning, and Basic Understanding factors predicted students' post-course sense of belonging. Basic Understanding and Productive Questioning also predicted future course selections. Interestingly, Perceived Effectiveness, which represents the core content of many current SET questionnaires, showed significant negative associations with course grades and future course choices. In light of the strong evidence emerging against current SETs, the behavioral focus in the current instrument offers a possible way forward. • SET instrument repeatedly samples in-class experiences. • The instrument focuses on observable instructor behaviors. • Prior, typical SET shows a negative relation to grades and future choices. • Active Learning positively predicts present and future grades. • Productive Questioning positively predicts future higher-level course selections.