Validation of a mini focus group interview for postgraduate year training program selection

A BSTRACT Objectives: The postgraduate year (PGY) general medical training program represents a major reform in Taiwan’s medical education. As PGY training expanded nationwide, hospitals were required to conduct entrance examinations; however, standardized selection tools remain limited. Since 2011, Kaohsiung Medical University Hospital (KMUH) has implemented a mini focus group interview for PGY recruitment. This study evaluates the reliability and validity of this interview method. Materials and Methods: A total of 408 applicants who participated in the KMUH PGY entrance examination during 2021–2022 were analyzed. Admission decisions weighted document review and interview performance equally (50% each). The interview consisted of 12 parallel structured assessment stations assessing three domains. Reliability was examined using Cronbach’s α and intraclass correlation coefficients (ICCs). Validity was assessed through rating consistency analysis and predictive validity between interview scores and case-based discussion (CbD) performance using Pearson’s correlation ( r ). Station equivalence was additionally evaluated using score distribution characteristics and the Kruskal–Wallis test. Results: The interview demonstrated excellent internal consistency (Cronbach’s α = 0.956). ICCs ranged from 0.581 to 0.750 and within-station Cronbach’s α ranged from 0.909 to 0.989. No significant differences were observed across the 12 parallel interview stations (Kruskal–Wallis H = 11.619, P = 0.393). Interview rankings showed moderate agreement with final PGY match outcomes ( κ = 0.477). The correlation between interview scores and CbD performance was r = 0.325 ( P < 0.05). The top 107 group had significantly higher CbD scores than the nontop 107 group (85.2 vs. 83.2, P < 0.05), suggesting acceptable predictive validity. Conclusion: The KMUH mini focus group interview demonstrates strong reliability, meaningful evidence of validity, comparable score distributions across parallel interview stations, and acceptable predictive validity, supporting its use as an effective PGY selection tool. Further multisite studies are warranted to confirm generalizability.