Assessing the Assessment: Utilization of a Year-Over-Year Longitudinal Approach for Improving the Quality of Early Stage Medical Student Examinations

Ensuring the quality and consistency of exam questions administered in pre-clerkship medical education is essential to the effective evaluation of first year medical students' foundational knowledge and readiness for advanced study. Based on the premise that early stage medical students are highly intelligent and motivated, we tested the hypothesis that poor performance on exam questions early in the medical education process reflected flaws in the instructional and/or assessment processes rather than student deficits. To validate this assumption, we employed a three-pronged, year-over-year longitudinal assessment model aimed at systematically improving the quality of exam questions for first-year students in the Musculoskeletal and Integument Module of the Uniformed Services University of the Health Sciences (USUHS) pre-clerkship curriculum. We focused on three key aspects in evaluating our summative exam questions: (1) comprehensive item analysis to ensure question reliability, validity, and an appropriate level of cognitive complexity; (2) troubleshooting sessions with teaching faculty and course directors to ensure instructional content and learning objectives maintained alignment with the exam questions under review; and (3) targeted use of formative knowledge check questions aimed at directing student mastery of relevant instructional content. Year-over-year longitudinal tracking of item performance analyzing metrics such as item difficulty, discrimination indices, and response patterns allowed us to identify inconsistencies between intended instructional delivery and student learning outcomes. The identification of questions that consistently underperformed facilitated targeted question review sessions that provided faculty the opportunity to analyze the clarity of question phrasing and content and to ensure alignment with their intended instructional content. Following item analysis and faculty review, we ascertained if the problem lay with the writing of the exam questions, the alignment of instructional content to exam questions, or insufficient instructional focus on the content. With respect to the latter, we also sought to determine if students would benefit from re-exposure to the information via formative knowledge check questions accompanied by detailed explanations. Question remediation targeting each of these aspects proved to be successful in improving performance in the subsequent year’s exams, with the use formative knowledge check questions proving to be especially effective. This year-over-year improvement in item performance confirmed an enhancement of student mastery, supporting the benefits of taking a dynamic longitudinal approach to evaluation of first year medical school assessments. Disclaimer: The opinions and assertions expressed herein are those of the author(s) and do not reflect the official policy or position of the Uniformed Services University of the Health Sciences or the Department of Defense. Uniformed Services University School of Medicine This abstract was presented at the American Physiology Summit 2025 and is only available in HTML format. There is no downloadable file or PDF version. The Physiology editorial board was not involved in the peer review process.