Do the numbers hold up? Grades, personality and the validity of student evaluations of teaching in an Omani foundation programme

Student evaluations of teaching (SET) carry high-stakes consequences, yet what they measure remains contested, and Arab Gulf evidence is scarce. Prompted by an institutional dispute in which teachers argued their ratings were unreliable and unfairly depressed by examination marks, this study examined the determinants and stability of SET in an Omani general foundation programme. A 40-item instrument, developed locally and supported by confirmatory factor analysis (four correlated factors; CFI = 0.87; composite reliabilities 0.85 to 0.97), was administered twice across the mid-term boundary to 411 students, 89 of them twice. Contrary to the teachers’ complaint, within-student ratings did not move once marks were known (|dz| = 0.02), and satisfaction with marks did not differentiate them. Yet the data resist a reassuring reading: perceived teacher personality was by far the strongest correlate of ratings and overlapped with them so heavily (r = 0.67) that the two are hard to separate; students’ own sense of evaluation responsibility and self-study skills contributed independently; and test–retest reliability over a median of 45 days was only 0.50 despite near-perfect stability in group means. Ratings showing no detectable effect of timing, mark satisfaction or gender can still fail to track teaching quality. Reliability is not validity.