Beyond the Mean: Evaluative Heterogeneity, Teaching Context, and the Evidential Meaning of Student Evaluations

Student evaluations of teaching (SETs) remain widely used in consequential institutional decisions, yet their interpretation typically relies on mean scores that obscure differences in student agreement and the contexts in which evaluations are produced. This study examines evaluation level and evaluative heterogeneity as distinct but complementary properties of SET evidence and investigates how each is structured across instructors, teaching contexts, and time. We analyze four years of university-wide administrative SET data using cross-classified and within-between models, complemented by context-adjusted institutional classifications. Evaluation level showed substantially greater instructor-associated clustering than heterogeneity, whereas heterogeneity was predominantly occasion-specific. Teaching conditions were differently associated with the two properties, and demographic associations were context dependent rather than uniform. Contextual adjustment rarely changed institutional classifications but mattered primarily near the decision boundary, where similar mean scores could rest on different evidential profiles. These findings show that a SET mean does not fully characterize the evidence supporting it. Response volume, evaluative heterogeneity, teaching context, and longitudinal stability can qualify the evidential weight attached to mean ratings, particularly when SETs inform consequential institutional decisions.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24336471
Primary Topic
Evaluation of Teaching Practices
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Beyond the Mean: Evaluative Heterogeneity, Teaching Context, and the Evidential Meaning of Student Evaluations

José María Barroso Tristán, Raquel Puente Castro, Miguel A. Pérez‐Uribe
Zenodo (CERN European Organization for Nuclear Research)
Evaluation of Teaching Practices
preprint

Beyond the Mean: Evaluative Heterogeneity, Teaching Context, and the Evidential Meaning of Student Evaluations

José María Barroso Tristán, Raquel Puente Castro, Miguel A. Pérez‐Uribe
preprint en

Abstract

Student evaluations of teaching (SETs) remain widely used in consequential institutional decisions, yet their interpretation typically relies on mean scores that obscure differences in student agreement and the contexts in which evaluations are produced. This study examines evaluation level and evaluative heterogeneity as distinct but complementary properties of SET evidence and investigates how each is structured across instructors, teaching contexts, and time. We analyze four years of university-wide administrative SET data using cross-classified and within-between models, complemented by context-adjusted institutional classifications. Evaluation level showed substantially greater instructor-associated clustering than heterogeneity, whereas heterogeneity was predominantly occasion-specific. Teaching conditions were differently associated with the two properties, and demographic associations were context dependent rather than uniform. Contextual adjustment rarely changed institutional classifications but mattered primarily near the decision boundary, where similar mean scores could rest on different evidential profiles. These findings show that a SET mean does not fully characterize the evidence supporting it. Response volume, evaluative heterogeneity, teaching context, and longitudinal stability can qualify the evidential weight attached to mean ratings, particularly when SETs inform consequential institutional decisions.

Zenodo (CERN European Organization for Nuclear Research)
Universidad Loyola Andalucía (ES), Universidad de Sevilla (ES), Tecnológico de Monterrey (MX)
Quality education
Evaluation of Teaching Practices
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon