Real-world assessment (RWA): validity evidence for cataract surgery with a tool your faculty will actually use

Background: We developed and implemented a real-world assessment tool (RWA) for resident surgery. It was designed to improve utility by reducing faculty and trainee cost (completion and training time, cognitive burden). RWA uses an entrustability scale where higher scores indicate a resident’s readiness for independent surgical practice. We describe a two-phase implementation of RWA and report validity evidence for cataract surgery assessment. Methods: RWA was developed iteratively with resident and faculty input and piloted at a single residency program in phase 1. Phase 2 implemented RWA at 11 ophthalmology residency programs. The main outcome measures were 1) number of surgical assessments submitted and 2) correlation between entrustment score on resident cataract surgery assessments and time in training. Results: After implementation of RWA, completed assessments increased for all participating programs. 1384 assessments of 111 residents by 112 faculty assessors were collected. Cross-sectional analysis of the assessments that examined cataract surgical procedural competency demonstrated that higher entrustment scores correlated with postgraduate year in training in both the single site (P= Conclusions: RWA demonstrates low cost, high acceptability, construct validity and reliability for cataract surgical skills assessment. Thus, RWA has shown high utility for assessment of cataract surgery. Because it is procedure agnostic it can be used to assess resident surgical skill for any procedure. RWA provides useful data for residency programs to monitor trainee development.