Pedagogical Primer: Messick's Unified Validity Framework

ABSTRACT Medical training relies heavily on assessment. Although gathering validity evidence for assessments is critical, validity is often treated as a fixed property that a test either possesses or lacks. Building on earlier work by Cronbach and Meehl, Messick reshaped this model by proposing that validity belongs to the interpretation and uses of scores rather than to the instrument itself. He also challenged the traditional division of validity into separate types, proposing instead that content, criterion, and construct evidence represent complementary sources of evidence that together support a single, unified view of construct validity, in which theory connects a score to the construct it is meant to represent and the evidence supports a judgment about whether a score means what it is intended to mean. This Pedagogical Primer traces the origins of that view, the developments that followed, and how it now guides assessment in medical education and research. The framework was developed out of decades of debate in educational measurement and was later codified in the Standards for Educational and Psychological Testing, which describe five sources of validity evidence: content, response processes, internal structure, relationship to other variables, and the consequences of testing. Kane offered a complementary approach that lays out the chain of inferences within a validity argument, so that the evidence gathered matches the specific claims a score is being asked to support. The unified validity framework provides educators with a way to plan the evidence needed before using an assessment to make consequential decisions and gives researchers a standard for reporting validation work without overreaching conclusions. This framework requires significant and sustained effort. The amount of evidence needed should be proportional to the stakes of the decision, although there is no fixed rule for when sufficient evidence has been gathered. Despite this, Messick's unified validity framework remains a practical guide for building and defending the assessments on which training decisions rest.