Domain-specific cognitive skill profiles in high-stakes tertiary assessment: a diagnostic model grounded in bloom’s taxonomy

High-stakes tests play a decisive role in test-takers’ lives and in shaping educational policies, yet they are often criticized for encouraging rote learning and limiting higher-order cognitive engagement. This study aimed to analyze the underlying cognitive processes measured in the domain-specific content knowledge section of the National PhD Entrance Exam (NPEE) in Teaching English as a Foreign Language (TEFL) in Iran, a high-stakes exam with far-reaching consequences. Employing a mixed-methods design, the quantitative phase analyzed item responses from 1000 examinees using the cognitive diagnostic model G-DINA, which was integrated with Bloom’s taxonomy of cognitive learning to investigate the underlying cognitive skills required to answer test items and provide a comprehensive understanding of the low- and higher-order cognitive levels of test-takers. In the qualitative phase, semi-structured interviews were conducted with PhD students and university professors from different universities. Results revealed that the test failed to measure cognitive skills beyond the executing level, corresponding to the third level of Bloom’s taxonomy. Findings further indicated that test items relied predominantly on recall of theories and factual details, creating a mismatch between the skills assessed and those required for TEFL PhD program real-life tasks. Stakeholder interviews emphasized that the exam underrepresents essential postgraduate competencies such as research ability, critical thinking, and problem solving. These shortcomings may result in negative washback effects by fostering teaching to the test, and undermining the quality of postgraduate TEFL research as well as deep learning in tertiary-level education.