Boosting liver segmentation accuracy with UNet++-ViT: integrating autoencoder and hybrid loss function

Abstract The presence of noise and intensity variations in medical images poses significant challenges to traditional segmentation methods. In this paper, we propose an enhanced UNet-based framework, UNet++-ViT, incorporating an autoencoder for image denoising and a hybrid loss function combining Tversky Loss and Focal Loss to improve segmentation performance. This paper introduces UNet++-ViT, a robust segmentation framework that integrates a Vision Transformer (ViT) for global contextual understanding, an autoencoder for effective denoising, and a hybrid Tversky-Focal loss to address class imbalance and refine boundary delineation. We evaluate the approach on the LiTS17 liver CT dataset and compare against the models using single loss functions. The proposed method demonstrates superior performance, achieving a Dice coefficient of 0.994, Jaccard index of 0.957, surpassing the leading model SwinUNet by 3.3 % & nnUNet by 4.1% in Dice score along with improved recall and specificity, outperforming other baseline approaches. These results confirm the effectiveness of combining denoising and hybrid loss strategies for robust and precise liver segmentation.