DETR-SAM: Combining Feature-Fused DETR Detector and Fine-Tuned SAM for Automatic Aortic Dissection Segmentation in CTA Image

Aortic Dissection (AD) is a life-threatening disease. As such, it’s vital to make clinical strategies by accurate segmentation of AD from Computed Tomography Angiography (CTA) images. Due to the heterogeneous morphology of AD in CTA images and the influence of image noise, inaccurate segmentation results are often produced, making it challenging to accurately identify the contours of AD. To meet these challenges, we propose a novel automatic AD segmentation model DETR-SAM based on the Segment Anything Model(SAM). Our model consists of a deformable DETR object detector module with multi-scale feature fusion and a fine-tuned SAM module for refining the segmentation contours. The object detector module effectively fuses the multi-scale image features extracted by the image encoder of SAM with the features obtained from the DETR backbone CNN network. The fused multi-scale features generate multiple prompts rich in geometric shape knowledge through multiple regression prediction heads to guide the SAM model to refine the segmentation boundaries and enhance the recognition of contours. Meanwhile, a fine-tuning strategy using these prompts is proposed to enhance the SAM model’s understanding of AD features. This strategy enables our DETR-SAM model with a 2D segmentation network architecture to surpass the performance of mainstream 3D segmentation algorithms. We have evaluated DETR-SAM on the ImageTBAD dataset and our internal dataset. The results showed that the accuracy of our model is better than the current state-of-the-art 2D and 3D segmentation methods in all four indicators. In addition, the effectiveness of the feature fusion module, hint generation method and fine-tuning strategy are further verified by ablation experiments. Our DETR-SAM method is helpful to a certain extent in the diagnosis and auxiliary treatment of AD in different clinical situations and lays a foundation for further research and clinical application.