Advanced Segmentation of Gastrointestinal (GI) Cancer Disease Using a Novel U-MaskNet Model

The purpose of this research is to contribute to the development of approaches for the classification and segmentation of various gastrointestinal (GI) cancer diseases, such as dyed lifted polyps, dyed resection margins, esophagitis, normal cecum, normal pylorus, normal Z line, polyps, and ulcerative colitis. This research is relevant and essential because of the current challenges related to the absence of efficient diagnostic tools for early diagnostics of GI cancers, which are fundamental for improving the diagnosis of these common diseases. To address the above challenges, we propose a new hybrid segmentation model, U-MaskNet, which is a combination of U-Net and Mask R-CNN models. Here, U-Net is utilized for pixel-wise classification and Mask R-CNN for instance segmentation, together forming a solution for classifying and segmenting GI cancer. The Kvasir dataset, which includes 8000 endoscopic images of various GI cancers, is utilized to validate the proposed methodology. The experimental results clearly demonstrated that the novel proposed model provided superior segmentation compared to other well-known models, such as DeepLabv3+, FCN, and DeepMask, as well as improved classification performance compared to state-of-the-art (SOTA) models, including LeNet-5, AlexNet, VGG-16, ResNet-50, and the Inception Network. The quantitative analysis revealed that our proposed model outperformed the other models, achieving a precision of 98.85%, recall of 98.49%, and F1 score of 98.68%. Additionally, the novel model achieved a Dice coefficient of 94.35% and IoU of 89.31%. Consequently, the developed model increased the accuracy and reliability in detecting and segmenting GI cancer, and it was proven that the proposed model can potentially be used for improving the diagnostic process and, consequently, patient care in the clinical environment. This work highlights the benefits of integrating the U-Net and Mask R-CNN models, opening the way for further research in medical image segmentation.