Developing and evaluating image-based metrics for open surgical suturing skill assessment

Abstract Purpose Open surgery training is vital in vascular surgery education. Traditionally, trainees’ open surgical suturing skills are evaluated by their educators, which is a subjective and time-consuming process. This paper evaluates image-based metrics for surgical suturing skill assessment by comparing performance among three groups: attending surgeons, resident surgeons, and novices. Methods A simulator, named SutureCoach, is developed to simulate a radial suturing task, performed either at “depth,” representing a cavity, or at “surface.” Among other sensors, the simulator includes cameras and computer vision software to track the motion of the needle and thread under the membrane being sutured. A set of image-based metrics are calculated for each suture based on video data. The set of metrics includes four previously proposed metrics as well as four new metrics inspired by a refined suturing phase segmentation scheme. To evaluate the performance of the image-based metrics, we conducted a study with 97 participants, divided almost evenly between attending surgeons, resident surgeons, and novices. Data was collected at several events over a 5-month period. Results The statistical analysis shows that all metrics have significantly different means when comparing performance between novices and surgeons at both conditions, surface and depth. Furthermore, seven out of eight metrics have significant mean differences when comparing resident surgeons’ and attending surgeons’ performance at the depth condition. Conclusion Image-based metrics are capable of suturing skill assessment, especially for distinguishing novices from surgeons. Depth constraints allow higher skill levels to be better differentiated.