Automated skill evaluation method for surgical images using optical flow and attention mechanism

This paper describes an automated skill evaluation method for surgical images based on optical flow and self-attention mechanisms. This method uses a segment of a surgical video as input to a deep learning network and outputs scores. Previous methods primarily analyzed the kinematic data of surgical robots to assess surgical skills; however, kinematic data is often unavailable in non-robotic surgeries. Therefore, we propose a regression model that uses surgical video as input data to evaluate surgical skills through optical flow and self-attention mechanisms. We use optical flow to extract motion information from surgical images and employ the ViViT (Video Vision Transformer) model to extract features from both image information and motion information. Finally, we concatenate the two feature vectors and output six scores corresponding to the GRS (Global Rating Scale) evaluation criteria. To validate the evaluation performance of the model, we calculated the RMSE (Root Mean Squared Error) between the model’s output scores of surgical videos and the ground truth scores in the surgical skill assessment dataset JIGSAWS. A low RMSE of 0.79 demonstrated that our method can reliably estimate scores, highlighting its potential for surgical skill assessment.