Segmentation and Classification in Radiological Images Using Deep Learning Approaches: A Comprehensive Comparison Study on Carotid Artery and Breast Lesions

Aim: This study aimed to comprehensively compare the performance of different deep learning architectures for automatic segmentation of carotid artery ultrasound images and classification of breast ultrasound images, and to develop clinical decision support systems.Material and Methods: Nine segmentation algorithms (U-Net, SegNet, DeepLab v3+, Mask R-CNN, Attention U-Net, U-Net++, TransUNet, Swin-UNet, Vision Mamba UNet) and ten classification algorithms were evaluated on a single held-out partition of the Common Carotid Artery (CCA; 1100 images from 11 subjects) and Breast Ultrasound Images (BUSI; 780 images) datasets; the segmentation models were trained with the configurations recommended in their original publications and the classification models with a shared SGD configuration. Performance was measured using the Dice similarity coefficient (DSC), intersection over union (IoU), precision, sensitivity and F1 score.Results: Vision Mamba UNet achieved the highest segmentation performance for carotid artery (DSC=0.943, IoU=0.896). For the BUSI dataset, Vision Mamba UNet obtained the highest segmentation DSC (0.834) and Vision Transformer the highest classification F1 (0.887). The three highest-ranked segmentation models differed by at most 0.005 DSC on CCA and 0.029 on BUSI.Conclusion: In this single-run evaluation, state space model-based and transformer-based segmentation architectures obtained higher scores than conventional CNN approaches; the ranking was not tested statistically. The web-based applications are prototypes that have not been evaluated in a clinical setting.