A ResNet-50–UNet Hybrid with Whale Optimization Algorithm for Accurate Liver Tumor Segmentation

<b>Objective:</b> Segmentation of liver and liver tumors from 3D medical images is a challenging and computationally expensive task. Organs that are in close proximity may have similar shape, texture, and intensity, which makes it difficult for accurate segmentation. Accurate segmentation of liver tumors is important for diagnosis and treatment planning of liver cancer. <b>Methods:</b> A hybrid model with a U-Net based structure and the Whale Optimization Algorithm (WOA) was proposed. WOA was used to optimize the hyperparameters of the conventional LiTS-Res-UNet to obtain the best segmentation performance of the deep learning model. <b>Results:</b> The LiTS-Res-Unet + WOA hybrid model achieved a performance of 99.54% for accuracy, with a Dice coefficient of 92.38% and a Jaccard index of 86.73% on the benchmark dataset, outperforming state-of-the-art methods. <b>Conclusions:</b> The WOA-based adaptive search space was able to obtain an optimal set of hyperparameters for deep learning model convergence while increasing the accuracy of the model in the proposed hybrid model. The robust performance and clinical applicability of the model in liver tumor segmentation were demonstrated.