Validity of the Family Medicine Certification Exam through the Online OSCE: A Cross-Sectional Study. (Preprint)

<sec> <title>BACKGROUND</title> Medical certification is a crucial mechanism for ensuring quality and safety in healthcare. In Mexico, the Mexican Board of Certification in Family Medicine (CMCMF) conducts certification through a two-phase process that includes an online Objective Structured Clinical Examination (OSCE). The shift to virtual environments after the COVID-19 pandemic has raised concerns about the validity and reliability of these assessments. </sec> <sec> <title>OBJECTIVE</title> To analyze the psychometric properties of the CMCMF online OSCE applied in the 65th (December 2023) and 66th (August 2024) certification exams. </sec> <sec> <title>METHODS</title> An observational, cross-sectional, descriptive study was conducted with 3,083 family physicians (1,797 from Cohort 65 and 1,286 from Cohort 66) who completed the practical phase of certification. Sociodemographic variables were described and compared across cohorts. Psychometric analyses included item discrimination using t-tests for extreme groups, reliability through Cronbach’s alpha, exploratory factor analysis with varimax rotation, and hierarchical multiple regression to assess predictive capacity. </sec> <sec> <title>RESULTS</title> Women represented 64.4% of the sample, with a higher proportion in Cohort 66 (p = 0.026). Item discrimination confirmed that the 26 items differentiated between high- and low-performing candidates (p &lt; 0.001). Cronbach’s alpha was 0.93 in both cohorts, indicating excellent internal consistency. Factor analysis identified four stable dimensions—preventive measures, clinical interview skills, doctor–patient relationship, and clinical interpretation skills—explaining 59% of the variance. Overall performance was comparable between cohorts (78.3 vs. 79.0). Hierarchical regression analysis showed that Factor 1 alone explained 70.2% of the variance; the inclusion of all factors increased the explained variance to R² = 1.000. </sec> <sec> <title>CONCLUSIONS</title> The CMCMF online OSCE demonstrated high reliability, a stable factor structure, and strong predictive capacity, supporting its validity as a tool for assessing clinical competencies in family medicine. Further research should evaluate its long-term applicability and generalization across specialties. </sec>