YOLO-UND: YOLO and Transformer-based ultrasound-guided puncture needle detection and localization

The accuracy and success rate of ultrasound-guided percutaneous puncture robot puncture seriously affect the efficacy and surgical safety. The aim of this study was to develop a puncture needle detection and localization method for US-guided puncture surgical robots. To detect multi-scale targets in real-time, we employ an efficient hybrid CNN-ViT backbone network. FasterViT replaces the pure convolutional backbone network in YOLOv8, which enhances the algorithm ability to acquire contextual information and helps improve feature extraction. In addition, we apply the multi-scale spatial attention method EPSANet on the feature maps extracted by the backbone network, which enables the network to pay attention to the multi-scale information in different regions of the feature maps, thus enhancing the model's ability to generalise and deal with complex contexts. In addition, we introduce skip connections for the Neck part of YOLO-UND, which enhances the feature fusion and multi-scale detection capabilities. Finally, we deployed the algorithm on a surgical robotic system and performed puncture needle localization experiments on the US model and different in vitro tissues. Our proposed method achieves real-time detection and localization of puncture needles with a localization error of 0.32 ± 0.15 mm and an inference time of 30 FPS, which demonstrates the good potential of our proposed method for clinical applications.