Beyond the Mean: Evaluative Heterogeneity, Teaching Context, and the Evidential Meaning of Student Evaluations
Student evaluations of teaching (SETs) remain widely used in consequential institutional decisions, yet their interpretation typically relies on mean scores that obscure differences in student agreement and the contexts in which evaluations are produced. This study examines evaluation level and evaluative heterogeneity as distinct but complementary properties of SET evidence and investigates how each is structured across instructors, teaching contexts, and time. We analyze four years of university-wide administrative SET data using cross-classified and within-between models, complemented by context-adjusted institutional classifications. Evaluation level showed substantially greater instructor-associated clustering than heterogeneity, whereas heterogeneity was predominantly occasion-specific. Teaching conditions were differently associated with the two properties, and demographic associations were context dependent rather than uniform. Contextual adjustment rarely changed institutional classifications but mattered primarily near the decision boundary, where similar mean scores could rest on different evidential profiles. These findings show that a SET mean does not fully characterize the evidence supporting it. Response volume, evaluative heterogeneity, teaching context, and longitudinal stability can qualify the evidential weight attached to mean ratings, particularly when SETs inform consequential institutional decisions.
Authors
- José María Barroso Tristán (ORCID: https://orcid.org/0000-0002-9190-5707)
- Raquel Puente Castro (ORCID: https://orcid.org/0000-0002-8902-1136)
- Miguel A. Pérez‐Uribe (ORCID: https://orcid.org/0000-0001-5348-1782)
Institutions
- Universidad Loyola Andalucía (ES)
- Universidad de Sevilla (ES)
- Tecnológico de Monterrey (MX)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24336471
- Primary Topic
- Evaluation of Teaching Practices
- Type
- preprint