Human-first learning: scaffolding reasoning before AI assistance improves diagnostic accuracy

Generative AI could improve physicians' clinical performance, yet its use in medical training raises questions about whether trainees can develop the clinical reasoning needed to use AI effectively and independently. We conducted a longitudinal randomised controlled trial involving 496 medical students (389 analysed). Students completed clinical reasoning tasks with virtual standardised patients and were randomised to six conditions: no intervention, an AI assistant alone, AI scaffolding alone, scaffolding followed by the AI assistant, the AI assistant followed by scaffolding, or both simultaneously. AI scaffolding prompted students to reflect on and evaluate their reasoning without providing direct answers; the assistant provided requested information and answers. The study comprised an AI-free baseline, two intervention phases and a final assessment after all AI support was withdrawn. Frontier large language models (LLMs) completed the same tasks as an additional benchmark. Students initially performed below LLMs but later exceeded them following the interventions. Students who began with AI scaffolding achieved higher diagnostic accuracy than those given access to the AI assistant from the outset both during the subsequent intervention phase (75.2% vs 66.0%; OR 1.73, 95% CIs [1.12–2.68]) and after withdrawal (24.6% vs 14.4%; OR 2.19, 95% CIs [1.15–4.14]). After withdrawal, scaffold-first groups were the only groups to outperform both unsupported students (24.5–24.7% vs 9.0%; ORs 3.77–3.87, 95% CIs [1.25–12.00]) and LLMs (24.5–24.7% vs 8.6%; ORs 3.48–3.50, 95% CIs [1.22–9.89]). Introducing the AI assistant after scaffolding decreased intrinsic cognitive load ( t (50) = 3.25, p BH = .006, Cohen’s d z = 0.46, 95% CIs [0.17, 0.75]) and increased metacognitive evaluation ( t (50) = 3.11, p BH = .019, Cohen’s d z = 0.44, 95% CIs [0.15, 0.73]). These findings support a staged and human-first approach to AI use in medical training, in which reasoning-focused scaffolding precedes answer-providing assistance to strengthen diagnostic performance both during AI-supported learning and after AI support is withdrawn.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-19
DOI
https://doi.org/10.21203/rs.3.rs-11417031/v1
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Human-first learning: scaffolding reasoning before AI assistance improves diagnostic accuracy

Jiaqi Xu, Yizhou Fan, Danijela Gašević, Oudong Xia et al.
Research Square
Artificial Intelligence in Healthcare and Education
preprint

Human-first learning: scaffolding reasoning before AI assistance improves diagnostic accuracy

Jiaqi Xu, Yizhou Fan, Danijela Gašević, Oudong Xia, Matthias J. Stadler, Dragan Gašević, Changmin Lin, Naping Chen, Taolin Zhu, Mingxue Xu, Luzhen Tang, Zijian Li, Mengyu Xia, Lixiang Yan, Xinyu Li, Linfei Xiao
preprint en

Abstract

Generative AI could improve physicians' clinical performance, yet its use in medical training raises questions about whether trainees can develop the clinical reasoning needed to use AI effectively and independently. We conducted a longitudinal randomised controlled trial involving 496 medical students (389 analysed). Students completed clinical reasoning tasks with virtual standardised patients and were randomised to six conditions: no intervention, an AI assistant alone, AI scaffolding alone, scaffolding followed by the AI assistant, the AI assistant followed by scaffolding, or both simultaneously. AI scaffolding prompted students to reflect on and evaluate their reasoning without providing direct answers; the assistant provided requested information and answers. The study comprised an AI-free baseline, two intervention phases and a final assessment after all AI support was withdrawn. Frontier large language models (LLMs) completed the same tasks as an additional benchmark. Students initially performed below LLMs but later exceeded them following the interventions. Students who began with AI scaffolding achieved higher diagnostic accuracy than those given access to the AI assistant from the outset both during the subsequent intervention phase (75.2% vs 66.0%; OR 1.73, 95% CIs [1.12–2.68]) and after withdrawal (24.6% vs 14.4%; OR 2.19, 95% CIs [1.15–4.14]). After withdrawal, scaffold-first groups were the only groups to outperform both unsupported students (24.5–24.7% vs 9.0%; ORs 3.77–3.87, 95% CIs [1.25–12.00]) and LLMs (24.5–24.7% vs 8.6%; ORs 3.48–3.50, 95% CIs [1.22–9.89]). Introducing the AI assistant after scaffolding decreased intrinsic cognitive load ( t (50) = 3.25, p BH = .006, Cohen’s d z = 0.46, 95% CIs [0.17, 0.75]) and increased metacognitive evaluation ( t (50) = 3.11, p BH = .019, Cohen’s d z = 0.44, 95% CIs [0.15, 0.73]). These findings support a staged and human-first approach to AI use in medical training, in which reasoning-focused scaffolding precedes answer-providing assistance to strengthen diagnostic performance both during AI-supported learning and after AI support is withdrawn.

Research Square
Peking University (CN), Shantou University (CN), Shantou University Medical College (CN), Monash University (AU), Technical University of Munich (DE), University of Hong Kong (HK), Tsinghua University (CN)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon