AI-based computer vision as an adjunct tool for CVS documentation in laparoscopic cholecystectomy

Abstract Background Despite widespread endorsement for Critical View of Safety (CVS) to prevent bile duct injury during laparoscopic cholecystectomy (LC), attainment remains low. Quality assurance is limited by inaccurate operative reports and absent visual documentation. This study evaluated a surgical artificial intelligence (AI) platform as an adjunct for CVS documentation, comparing its performance to narrative operative reports using expert video review as the reference-standard. Methods This retrospective study included consecutive LC videos routinely captured and automatically analyzed by an implemented AI-based computer vision platform in a tertiary center (December 2022-August 2023). Three expert surgeons independently double-reviewed all videos, establishing ground-truth for CVS achievement by majority consensus. CVS classifications by the AI-platform and operative reports were compared against the ground-truth. Performance metrics were calculated and discriminatory performance was examined, due to class imbalance, using the area under the receiver operating characteristic curve (AUROC) and Matthews Correlation Coefficient (MCC). Analyses were stratified according to Parkland’s grading scale. Results Among 279 LC videos, 83.5% involved high disease severity (Parkland 3–5). The overall intra-rater reliability was high (Cohen’s kappa > 0.8), and multi-rater reliability was moderate (Fleiss’ kappa = 0.58). Ground-truth indicated CVS achievement in 74.2% of cases, compared with 55.2% identified by the AI-platform and 92.5% documented in operative reports. Despite similar overall agreement (73.8% vs. 73.8%; p = 0.39), the AI-platform had higher reliability (κ 0.45 vs. 0.13; p < 0.001), reflecting differing sensitivity (69.6% vs. 99.0%) and specificity (86.1% vs. 10.6%). In the context of class imbalance, the AI-platform had significantly higher AUROC than operative reports (0.77 vs. 0.55; p < 0.001), with consistently superior discriminatory CVS documentation across disease severity. MCC was 0.49 for the AI-platform and 0.13 for operative reports. Conclusions The AI-platform appears to have superior discriminatory performance and more reliable alignment with expert ground-truth than narrative documentation, supporting its use as an adjunct for CVS documentation.