Automated machine learning assessment and feedback for the Fundamentals of Laparoscopic Surgery peg transfer task
Aim: Whereas computer vision has advanced the capabilities of Fundamentals of Laparoscopic Surgery (FLS) trainers, real-time coaching using machine learning (ML) remains limited. This retrospective proof-of-concept study evaluated whether an ML model could provide automated FLS pass/fail assessment comparable to faculty, using pre-recorded videos. Methods: Residents performed video-recorded FLS peg transfers scored pass/fail. A training set of 60 videos (51 standard cases, 9 edge-case videos) was used to develop a multi-stage pipeline combining a “you only look once” (YOLOv8n) object detector, and a deterministic rule engine encoding FLS pass/fail criteria. The model was evaluated on 24 resident-disjoint, held-out videos. A faculty majority-vote reference standard was set by five evaluators per video. Model predictions were compared against this reference standard. Results: The ML model classified outcomes accurately in 92% of cases [95%CI: 74%-98%; F1 score = 0.93 (pass positive); Cohen’s κ = 0.83]. Total analysis time was 6 min vs. approximately 56 min of video playback. The model concurrently identified errors in 70% (31/44) of error events, with 97% (30/31) concordance in error classification. Among the 13 remaining events, the model preceded the faculty-annotated timestamp in 8/13 (62%). Real-time instructional prompts were implemented in a heads-up display but not evaluated here. Conclusion: The system provided accurate FLS peg-transfer assessment comparable to faculty while reducing evaluation time. As a retrospective proof-of-concept, it supports automated assessment and, pending prospective validation, could deliver scalable real-time feedback in skills training. External validation in larger, multi-institutional cohorts should precede adoption.
Authors
- Issam Koleilat (ORCID: https://orcid.org/0000-0001-8103-009X)
- Philip S. Barie (ORCID: https://orcid.org/0000-0002-5848-527X)
- Joelle Getrajdman
- Mayur Narayan (ORCID: https://orcid.org/0000-0002-7232-7446)
- Advaith Bongu (ORCID: https://orcid.org/0009-0009-5339-3139)
- Amin Madani (ORCID: https://orcid.org/0000-0003-0901-9851)
- Yunzhe Xue
- Usman Roshan
- Divya Kewalramani (ORCID: https://orcid.org/0009-0004-9125-0909)
- Nell Maloney Patel (ORCID: https://orcid.org/0000-0003-2310-5094)
- Andrew Hu (ORCID: https://orcid.org/0009-0005-3487-3787)
- Finn Kliewer
- Jenny Cai (ORCID: https://orcid.org/0000-0002-1808-8712)
- Ricardo Nunez
Institutions
- Johnson University (US)
Publication Details
- Journal
- Artificial Intelligence Surgery
- Published
- 2026-11-19
- DOI
- https://doi.org/10.20517/ais.2026.72
- Primary Topic
- Surgical Simulation and Training
- Type
- article
- Field-Weighted Citation Impact
- 0.00