Establishing a pass/fail standard for VR simulator based full crown preparation: a validity study using the contrasting groups method

Tooth preparation for a maxillary central incisor full crown is a fundamental but technically demanding psychomotor skill in prosthodontics. Virtual reality (VR) simulators can provide standardized and objective performance assessment; however, evidence-based pass/fail standards for simulator-based proficiency remain limited. Sixty-six participants were enrolled, including novices ( n = 23), intermediates ( n = 33), and experts ( n = 10). Each participant completed two consecutive full-crown preparations on a VR simulator, with automated feedback after the first attempt. Thirteen simulator metrics were initially evaluated, and 12 were retained according to predefined item-analysis criteria. Group discrimination and immediate within-simulator changes were assessed. The pass/fail standard was established using the contrasting groups method and examined through observed classification outcomes. The 12-metric composite score showed internal consistency (Cronbach’s α = 0.89). First-attempt scores differed among the three experience groups (F (2,63) = 9.02, P < 0.001, η²= 0.223). Scores increased after feedback in all groups; mean changes were 6.77 points in novices, 4.69 in intermediates, and 5.16 in experts. The intersection of the fitted novice and expert normal density functions was 74.36 points. At the operational threshold of 78 points, 1 of 23 novices passed (false-positive rate, 4.3%; exact 95% CI, 0.1%–22.0%) and 3 of 10 experts failed (false-negative rate, 30.0%; exact 95% CI, 6.7%–65.2%). At 80 points, the novice false-positive rate remained 4.3%, while the expert false-negative rate increased to 50.0%. The findings provide preliminary validity evidence for an objective, simulator-based assessment benchmark for maxillary central incisor full-crown preparation. The 78-point threshold may support progression decisions within simulation-based training, but it should not be used as a standalone clinical credential. Predictive validity for subsequent clinical performance remains to be established.

Authors

Institutions

Publication Details

Journal
BMC Medical Education
Published
2026-11-21
DOI
https://doi.org/10.1186/s12909-026-10875-7
Primary Topic
Surgical Simulation and Training
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Establishing a pass/fail standard for VR simulator based full crown preparation: a validity study using the contrasting groups method

袁华, 程洁, Fengqing Chu, Gang Chen et al.
BMC Medical Education
Surgical Simulation and Training
article

Establishing a pass/fail standard for VR simulator based full crown preparation: a validity study using the contrasting groups method

袁华, 程洁, Fengqing Chu, Gang Chen, Xiaoqing Lu, Yu Fu, Jiajia Jiang, Hongbing Jiang
article en

Abstract

Tooth preparation for a maxillary central incisor full crown is a fundamental but technically demanding psychomotor skill in prosthodontics. Virtual reality (VR) simulators can provide standardized and objective performance assessment; however, evidence-based pass/fail standards for simulator-based proficiency remain limited. Sixty-six participants were enrolled, including novices ( n = 23), intermediates ( n = 33), and experts ( n = 10). Each participant completed two consecutive full-crown preparations on a VR simulator, with automated feedback after the first attempt. Thirteen simulator metrics were initially evaluated, and 12 were retained according to predefined item-analysis criteria. Group discrimination and immediate within-simulator changes were assessed. The pass/fail standard was established using the contrasting groups method and examined through observed classification outcomes. The 12-metric composite score showed internal consistency (Cronbach’s α = 0.89). First-attempt scores differed among the three experience groups (F (2,63) = 9.02, P < 0.001, η²= 0.223). Scores increased after feedback in all groups; mean changes were 6.77 points in novices, 4.69 in intermediates, and 5.16 in experts. The intersection of the fitted novice and expert normal density functions was 74.36 points. At the operational threshold of 78 points, 1 of 23 novices passed (false-positive rate, 4.3%; exact 95% CI, 0.1%–22.0%) and 3 of 10 experts failed (false-negative rate, 30.0%; exact 95% CI, 6.7%–65.2%). At 80 points, the novice false-positive rate remained 4.3%, while the expert false-negative rate increased to 50.0%. The findings provide preliminary validity evidence for an objective, simulator-based assessment benchmark for maxillary central incisor full-crown preparation. The 78-point threshold may support progression decisions within simulation-based training, but it should not be used as a standalone clinical credential. Predictive validity for subsequent clinical performance remains to be established.

BMC Medical Education
Stomatology Hospital (CN), Second Affiliated Hospital of Nanjing Medical University (CN), Nanjing Medical University (CN)
Surgical Simulation and Training
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon