Shared Spatiotemporal Multitask Learning for Simultaneous Surgical Gesture Recognition and Skill Assessment from Minimally Invasive Surgery Videos

The automatic recognition of surgical gestures and the assessment of surgical skills are essential for improving patient safety and optimizing surgical training processes. In this paper, a shared spatiotemporal multitask learning framework for joint surgical gesture recognition and skill assessment from minimally invasive surgery videos is presented. The proposed architecture simultaneously recognizes surgical gestures and estimates the expertise level of the surgeon from surgical video sequences. It is based on a 3D CNN with two parallel branches based on the ResNet18 backbone and incorporates LSTM module to model spatial temporal features from surgical video sequences. In addition, fully connected layers are used to perform surgical gesture classification in one branch and surgical skill level estimation. The experimental evaluation includes surgical gesture recognition and skill assessment using both segmented video clips and full-length surgical videos. The performance of the method is evaluated with the accuracy, precision, recall, and F1-score, for both tasks. These findings suggest that shared spatiotemporal feature learning provides an effective approach for jointly analyzing surgical gestures and skill levels from video.

Authors

Institutions

Publication Details

Journal
Sci
Published
2026-11-12
DOI
https://doi.org/10.3390/sci8110369
Primary Topic
Surgical Simulation and Training
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Shared Spatiotemporal Multitask Learning for Simultaneous Surgical Gesture Recognition and Skill Assessment from Minimally Invasive Surgery Videos

Juan C. Sánchez, Victor Javier Gonzalez-Villela, Daniel Haro‐Mendoza, Rodrigo Eduardo Arevalo-Ancona et al.
Sci
Surgical Simulation and Training
article

Shared Spatiotemporal Multitask Learning for Simultaneous Surgical Gesture Recognition and Skill Assessment from Minimally Invasive Surgery Videos

Juan C. Sánchez, Victor Javier Gonzalez-Villela, Daniel Haro‐Mendoza, Rodrigo Eduardo Arevalo-Ancona, Manuel Cedillo-Hernández
article en

Abstract

The automatic recognition of surgical gestures and the assessment of surgical skills are essential for improving patient safety and optimizing surgical training processes. In this paper, a shared spatiotemporal multitask learning framework for joint surgical gesture recognition and skill assessment from minimally invasive surgery videos is presented. The proposed architecture simultaneously recognizes surgical gestures and estimates the expertise level of the surgeon from surgical video sequences. It is based on a 3D CNN with two parallel branches based on the ResNet18 backbone and incorporates LSTM module to model spatial temporal features from surgical video sequences. In addition, fully connected layers are used to perform surgical gesture classification in one branch and surgical skill level estimation. The experimental evaluation includes surgical gesture recognition and skill assessment using both segmented video clips and full-length surgical videos. The performance of the method is evaluated with the accuracy, precision, recall, and F1-score, for both tasks. These findings suggest that shared spatiotemporal feature learning provides an effective approach for jointly analyzing surgical gestures and skill levels from video.

SciVol. 8(11)
Instituto Politécnico Nacional (MX), Universidad Nacional Autónoma de México (MX)
Instituto Politécnico Nacional, Universidad Nacional Autónoma de México
Good health and well-being
Surgical Simulation and Training
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon