Using distilled knowledge from large transformer models for efficient identification of cataract surgery steps

Introduction Automated analysis of cataract surgery videos can provide objective performance metrics for trainees and support consistent progress tracking. Large vision transformers perform well for medical image tasks, but their size can limit practical deployment. Methods We evaluated whether knowledge distillation can produce an efficient model for cataract surgical step recognition while preserving accuracy. We analyzed 268 resident cataract surgery videos collected across six sites and labeled with 14 surgical steps. A large teacher model pretrained with self-supervision was compared with a smaller student model trained using knowledge distillation, the same student trained without distillation, and a convolutional neural network baseline. Performance was assessed using overall accuracy and area under the receiver operating characteristic curve, and efficiency was assessed using parameter count and floating-point operations per frame. Results The distilled student achieved 87.7% accuracy and an area under the curve of 0.985, comparable to the teacher (87.7% accuracy; 0.970). The distilled student was substantially smaller (21.7 million vs. 85.8 million parameters) and more efficient (4.63 vs. 17.58 GFLOPs per frame). It also outperformed the student trained without distillation (85.5% accuracy; 0.963) and the best convolutional neural network baseline (85.6% accuracy; 0.973). Discussion These findings suggest that knowledge distillation can produce a lightweight, high-performing model for cataract surgical step recognition, reducing computational barriers relevant to deployment of automated feedback tools for surgical training.

Authors

Institutions

Publication Details

Journal
Frontiers in Digital Health
Published
2026-11-19
DOI
https://doi.org/10.3389/fdgth.2026.1869080
Primary Topic
Surgical Simulation and Training
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Using distilled knowledge from large transformer models for efficient identification of cataract surgery steps

Sophia Ying Wang, John Xiang
Frontiers in Digital Health
Surgical Simulation and Training
article

Using distilled knowledge from large transformer models for efficient identification of cataract surgery steps

Sophia Ying Wang, John Xiang
article en

Abstract

Introduction Automated analysis of cataract surgery videos can provide objective performance metrics for trainees and support consistent progress tracking. Large vision transformers perform well for medical image tasks, but their size can limit practical deployment. Methods We evaluated whether knowledge distillation can produce an efficient model for cataract surgical step recognition while preserving accuracy. We analyzed 268 resident cataract surgery videos collected across six sites and labeled with 14 surgical steps. A large teacher model pretrained with self-supervision was compared with a smaller student model trained using knowledge distillation, the same student trained without distillation, and a convolutional neural network baseline. Performance was assessed using overall accuracy and area under the receiver operating characteristic curve, and efficiency was assessed using parameter count and floating-point operations per frame. Results The distilled student achieved 87.7% accuracy and an area under the curve of 0.985, comparable to the teacher (87.7% accuracy; 0.970). The distilled student was substantially smaller (21.7 million vs. 85.8 million parameters) and more efficient (4.63 vs. 17.58 GFLOPs per frame). It also outperformed the student trained without distillation (85.5% accuracy; 0.963) and the best convolutional neural network baseline (85.6% accuracy; 0.973). Discussion These findings suggest that knowledge distillation can produce a lightweight, high-performing model for cataract surgical step recognition, reducing computational barriers relevant to deployment of automated feedback tools for surgical training.

Frontiers in Digital HealthVol. 8
Stanford University (US)
Surgical Simulation and Training
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon