Assessing the Quality of Objective Structured Clinical Examination (OSCE) Stations in Pharmacy Education: A Study from King Abdulaziz University

Abstract Context: Objective structured clinical examinations (OSCEs) are widely used in pharmacy education, yet few studies evaluate their station-level psychometric properties. This study applied a multi-metric framework aligned with the Association for Medical Education in Europe Guide No. 49 to assess OSCE quality. Aims: The aim of the study was to evaluate the psychometric performance—reliability, criterion-related validity, discriminatory capacity, and between-group variation—of OSCE stations in a PharmD programme. Settings and Design: A cross-sectional study was conducted at a large academic institution using OSCE data from 4th- and 5th-year PharmD students. Materials and Methods: Data were collected from 141 4th-year and 131 5th-year students during the 2024 academic year. Psychometric evaluation included internal consistency (Cronbach’s α ), criterion-related validity ( R ²), inter-grade discrimination (slope between checklist scores and global ratings), and between-group variation (one-way analysis of variance [ANOVA]). Station performance was analyzed across nine clinical domains. Statistical Analysis Used: Descriptive statistics, one-way ANOVA, and linear regression were used to examine psychometric indicators. Results: Fourth-year Station 3 demonstrated the strongest performance ( α = 0.70; R ² = 0.78; discrimination = 1.85), while Station 2 showed weaker metrics. In the 5th year, Station 1 showed the most robust indicators ( α = 0.80; R ² = 0.71), whereas Station 3 underperformed. Between-group variation was non-significant ( P > 0.05) except in fourth-year Station 1 ( P = 0.006). Communication and ethics scored highest, while patient education and laboratory data interpretation were lowest. Conclusions: OSCE performance varied across stations and domains, with examiner and station effects contributing to score differences. Structured calibration and targeted station redesign are recommended to enhance reliability, fairness, and curricular alignment.