Hybrid U-Net Model with Transformer-Enhanced Skip Connections for Abdominal Organ Segmentation in Laparoscopic Images

Abstract The organ Segmentation in surgical images plays a crucial role in computer-assisted diagnosis and robotic surgeries, yet it faces challenges like class imbalance, occlusions, and computational constraints. In order to overcome these challenges, we propose an architecture which uses asymmetric multi-scale transformer along with the attention-gated skip connections in U-Net Model to efficiently segment abdominal organs. Our proposed architecture allows efficient extraction of local features and modelling of global contexts. We also propose a hybrid loss function by combining Lovasz-Softmax loss and sparse categorical cross-entropy loss, which helps in overcoming class imbalance in the training process and improving performance on the minor classes. Our proposed method attains a mean precision of 0.9667, mean recall of 0.9432, mean dice of 0.9522, mean IoU of 0.9150, and mean HD95 of 0.4704 on all 13 classes of the CholecSeg8k dataset, which makes it better than the state-of-the-art techniques.