Automated Segmentation and Classification of Liver tumours in CT Images Using a Hybrid Transformer-CNN Framework with Metaheuristic Optimization

Liver cancer, particularly Hepato Cellular Carcinoma (HCC), remains one of the leading causes of cancer-related mortality worldwide. Early detection and accurate characterization of liver tumours are critical for effective treatment planning and improved patient outcomes. This paper presents a comprehensive Deep Learning (DL)based framework for the automatic segmentation and classification of liver tumours from Computed Tomography (CT) scan images. The developed methodology integrates an optimized preprocessing technique—African Vulture Optimization based Adaptive Histogram Equalization (AVO-AHE) algorithm to improve image quality and tumour visibility. For segmentation, a novel hybrid model, TransDenseU-Net, which combines the strengths of Convolutional Neural Network (CNN) and Transformers is developed for better accuracy in detecting complex and irregular tumour structures. Furthermore, a Deep Convolutional Neural Network (Deep CNN) utilizing a pretrained VGG-16 model is employed for tumour classification, distinguishing between malignant and benign cases. Classification performance is further enhanced by integrating Double Exponential Smoothing (DES) into the Marine Predator Algorithm (MPA) for optimized network training. The developed models demonstrate superior performance over conventional approaches, offering a scalable, accurate, and automated solution for liver tumour diagnosis and aiding clinicians in early decision-making.