Transformer-Based Automatic Pipeline for 3D Needle Localization on Intra-Procedural 3D MRI

Motivation: Needle localization on 3D magnetic resonance imaging (MRI) is critical for MRI-guided percutaneous interventions, but current manual methods are time-consuming. Goal(s): To develop a transformer-based pipeline for accurate and rapid automatic 3D needle localization on intra-procedural 3D MRI. Approach: The proposed pipeline adopted a coarse-to-fine segmentation strategy by combining 3D and 2D shifted window (Swin) Transformer networks. The performance was evaluated in pre-clinical pig datasets and compared with human-annotated references. Results: Computation time was 6 sec/volume. The median 3D needle tip and axis localization errors were 1.48 mm (1.09 pixels) and 0.98°, which were comparable to human-level accuracy. Impact: The automatic transformer-based pipeline developed in this work achieved rapid (~6s) and accurate pixel-level 3D needle localization on intra-procedural 3D MRI. This new pipeline has the potential to improve MRI-guided percutaneous interventions.