Real-time instance segmentation of spine endoscopy images using a YOLO-V11 deep convolutional neural network
BACKGROUND AND OBJECTIVES: Spine endoscopic surgery requires precise anatomical recognition and real-time bleeding detection to enhance surgical safety and efficiency. However, most existing computer vision studies do not offer real-time multi-class segmentation of the entire scene. This study aims to develop a real-time instance segmentation model based on YOLO-V11, capable of automatically identifying surgical instruments, critical anatomical structures, and bleeding sites in spine endoscopy. METHODS: The model was trained and evaluated using 2,181 annotated frames (training 1,741 / validation 216 / held-out test 224) from spine endoscopy images, covering nine key categories: instruments, bone, ligamentum flavum, dura, soft tissue, vessels, fat, the radiofrequency ablation wand, and bleeding focus. The segmentation model's performance was evaluated across various YOLO-V11 variants. RESULTS: Across five random seeds per variant on the same NVIDIA A100 GPU, YOLO-V11 l achieved the highest mask mAP50, defined as mean average precision at an intersection-over-union threshold of 0.50, with a value of 0.7735 ± 0.0030 (mean ± SD, n = 5). It also maintained real-time inference, reaching 270 ± 35 frames per second (FPS) with a per-frame latency of 3.75 ± 0.51 ms. All five variants exceeded 30 FPS, the widely accepted real-time threshold for surgical imaging. The model demonstrated high precision in segmenting vital structures such as the dura and ligamentum flavum and accurately identified bleeding regions. CONCLUSION: This research contributes to AI-driven surgical guidance systems, with potential applications in robotic-assisted spine surgery. The developed model enables real-time navigation, improving intraoperative decision-making and enhancing surgical outcomes. In particular, our YOLO-V11-based framework extends prior single-structure or lower-frame-rate multi-element approaches by delivering real-time multi-class instance segmentation of instruments, anatomical structures, and bleeding sites in spine endoscopy, with explicitly quantified inference latency.
Authors
- Jiho Kang (ORCID: https://orcid.org/0000-0001-7789-9251)
- Taeshin Kim (ORCID: https://orcid.org/0000-0002-7078-4387)
- Y. K. KIM C. S. LIM (ORCID: https://orcid.org/0009-0007-4450-1420)
- Wonchang Choi (ORCID: https://orcid.org/0009-0004-5763-1935)
- Hyunjae Kim (ORCID: https://orcid.org/0009-0007-3255-0639)
- Chulmin Lee (ORCID: https://orcid.org/0009-0001-6773-2412)
- Noah Euyhyun Nam
- Sang Kyeong Lee
- Hyun Ji Jung (ORCID: https://orcid.org/0009-0000-5815-9047)
- Gunyul Kim
- Junsu Lee (ORCID: https://orcid.org/0009-0001-2708-4780)
Institutions
- Sogang University (KR)
- Konyang University (KR)
- Korea University (KR)
- Soonchunhyang University (KR)
- Soonchunhyang University Hospital Cheonan (KR)
- Samsung Electronics (South Korea) (KR)
- Pusan National University (KR)
- Chung-Ang University (KR)
Publication Details
- Journal
- PLoS ONE
- Published
- 2026-11-18
- DOI
- https://doi.org/10.1371/journal.pone.0361612
- Primary Topic
- Advanced Neural Network Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00