Detection of Surgical Instruments Based on Synthetic Training Data

Due to a significant shortage of healthcare staff, medical facilities are increasingly challenged by the need to deploy current staff more intensively, which can lead to significant complications for patients and staff. Digital surgical assistance systems that track all instruments used in procedures can make a significant contribution to relieving the load on staff, increasing efficiency, avoiding errors and improving hygiene. Due to data safety concerns, laborious data annotation and the complexity of the scenes, as well as to increase prediction accuracy, the provision of synthetic data is key to enabling the wide use of artificial intelligence for object recognition and tracking in OR settings. In this study, a synthetic data generation pipeline is introduced for the detection of eight surgical instruments during open surgery. Using 3D models of the instruments, synthetic datasets consisting of color images and annotations were created. These datasets were used to train common object detection networks (YOLOv8) and compared against networks solely trained on real data. The comparison, conducted on two real image datasets with varying complexity, revealed that networks trained on synthetic data demonstrated better generalization capabilities. A sensitivity analysis showed that synthetic data-trained networks could detect surgical instruments even at higher occlusion levels than real data-trained networks. Additionally, 1920 datasets were generated using different parameter combinations to evaluate the impact of various settings on detection performance. Key findings include the importance of object visibility, occlusion, and the inclusion of occlusion objects in improving detection accuracy. The results highlight the potential of synthetic datasets to simulate real-world conditions, enhance network generalization, and address data shortages in specialized domains like surgical instrument detection.