Endoscopic sleeve gastroplasty video annotation: analysis of the learning curve of three annotators to achieve procedure proficiency

Introduction: Endoscopic interventional skills acquisition generally relies on apprenticeship-based training. Limitations of this learning model include the availability of trainers and human-factor biases. Increased routine surgical and endoscopic video recording and storage promote video-based learning (VBL) methods. VBL could shorten the learning curve by increasing trainer availability and democratizing training access for procedures such as endoscopic sleeve gastroplasty (ESG), which is carried out by a few expert operators. The aim of this work was to evaluate video annotation as a learning tool for ESG. Methods: Videos of ESG performed at a single center were collected and annotated. An annotation manual developed by an ESG expert was used by three annotators with different levels of endoscopic and ESG expertise. Pairwise Cohen’s kappa was used to assess inter-rater reliability (IRR). Completion of the learning curve for each phase and feature of ESG was calculated as the probability of accordance among annotators and estimated by a moving average optimization model with an expectation-maximization algorithm. Results: In total, 40 ESG videos were annotated. IRR pairwise kappa scores ranged between 0.44 (Annotator 1 and 3) and 0.58 (Annotator 2 and 3). Learning curve analysis showed that a different number of procedures need to be annotated to master the various phases of ESG, with the “other suture phase” needing the highest number of procedures ( n = 36). The overall learning curve showed variability across different aspects of ESG, with an overall average of 28.40 procedures needed to achieve proficiency. Conclusions: The learning process differs among trainees, as mirrored in the variable number of procedures needed to master ESG. Tailoring training for different trainees could lead to the shortening of learning curves. Some features of ESG are mastered more quickly; however, careful and repeated practice is essential for minimizing errors in more complex features. VBL could play a role in the democratization of high-level learning.