Concept before capability - a validity-led approach to AI in OSCEs

Objective Structured Clinical Examinations (OSCEs) remain central to the assessment of clinical competence, yet the integration of artificial intelligence (AI) into assessment design, delivery, and scoring raises a fundamental question: what evidence of competence are OSCEs intended to generate? This paper argues that AI should be evaluated through the lens of OSCE validity, with particular attention to how it may strengthen or weaken the interpretive chain linking observed performance to assessment decisions. Drawing on Messick's unitary conception of validity and Kane's argument-based framework, we examine how AI may introduce distinct threats across the scoring, generalisation, extrapolation, and implication inferences that underpin OSCE interpretation. We argue that generative AI (genAI) creates a particular risk of construct drift, whereby plausible assessment materials or scoring processes gradually reward observable proxies rather than the underlying clinical capabilities they are intended to assess. To translate this validity perspective into practice, we propose a framework that maps AI applications across the OSCE workflow according to their proximity to candidate consequences and the evidential burden required to justify their use. Lower-risk applications, such as design support and post-hoc analytics, may strengthen assessment when accompanied by construct-focused review and human oversight, whereas higher-risk uses, including AI-mediated candidate encounters and automated summative scoring, require substantially stronger validity evidence. We conclude that AI should be adopted according to the principle of concept before capability: assessment communities must first establish what clinical competence means and what evidence should count before deciding where AI has a legitimate role in high-stakes assessment.