Supporting Reliable Assessment of Clinical Judgment Skills in Simulation Environments

Background: Clinical judgment skills, a key component of nursing competency, are difficult to accurately and consistently assess. To support evaluation of these skills using Lasater's ( 2007 ) clinical judgment rubric (LCJR), rater training resources were developed and evaluated. Method: A one-group pretest-posttest design evaluated the effect of developed resources on agreement between two novice raters (NRs) and resource developers. NR-CE had content expertise; NR-NNE had no nursing experience. Results: Pre-training, total scores showed poor agreement between novice and trained raters (ICC absolute (2,1) LCJR total for NR-CE = −.59; LCJR total for NR-N NE = .24). Posttraining, agreement was higher (ICC absolute (2,1) LCJR total NR- CE = .70; LCJR total NR-N NE = .76). Dimension-level scoring showed more variability in reliability estimates. Conclusion: Training supported raters with and without content expertise in developing a shared mental model of CJS. Future work should replicate these findings with larger samples for more confidence in results and examine convergent validity by comparing ratings to external benchmarks.