Response Format and Construct Validity in Medical Students' Word‑Analysis Assessment: A Validity Argument Approach

Response format may influence the interpretation of language assessment scores by engaging different cognitive processes. This study aimed to examine the effects of response format on medical students' word-analysis performance. It further investigated the degree of construct overlap between the two formats and the potential moderating role of gender. A within-subjects counterbalanced comparative assessment design was employed with 170 undergraduate medical students who completed both a Multiple-Choice Word Analysis Test (MCQ) and a Word Analysis Completion Test (WACT). Data were analyzed using descriptive and inferential statistical procedures. Participants scored significantly higher on the MCQ (M = 14.61, SD = 3.60) than on the WACT (M = 7.31, SD = 4.39, p < .001). A moderate positive correlation was observed between the two formats (ρ = .42, p < .001), indicating shared but distinct construct variance. Gender did not significantly moderate this relationship (p = .420). Interpreted through Kane's argument-based validity framework, the findings demonstrated that response format constitutes an important component of construct representation and influences the interpretation of assessment scores. The findings suggest that recognition-based and production-based assessments capture complementary aspects of morphological competence. A mixed-format assessment approach may therefore provide more valid and comprehensive evidence of medical students' word-analysis ability.