A pilot study of real-world operative video analysis for assessing non-technical skills in surgical trainees

Abstract Cognitive non-technical skills (NTS), encompassing situation awareness and decision-making, are critical to safe surgical practice and frequently implicated in adverse events. Data-driven approaches offer opportunities to provide objective, reproducible evaluation of cognitive NTS (e.g. through sensor use). While operative video analysis is increasingly applied to assess technical skills, its role in assessing cognitive NTS remains underexplored. This study aims to evaluate the feasibility, validity and educational utility of operative video analysis for assessing surgical trainees’ cognitive NTS. This will be a single-center, longitudinal observational cohort study at a tertiary teaching hospital in Scotland. Eligible trainees performing laparoscopic appendicectomy and cholecystectomy will undergo video capture of both the operative field and the operating room. NTS will be rated using the Non-Technical Skills for Surgeons (NOTSS) framework. Video annotations will generate metrics reflecting cognitive processes, which will be correlated with NOTSS scores. Individualized feedback reports containing video-derived metrics will be provided to trainees. Acceptability and perceived educational utility of these reports will be evaluated through trainee and trainer surveys. Statistical analysis will include inter-rater reliability, correlation and receiver operating characteristic analyses. Formal ethical approval for this study has been granted. Results will be disseminated through conferences and peer-reviewed publication.