U-Net and Watershed Based Deep Learning Approach for Panoptic Segmentation of Colorectal Polyps in Colonoscopy

Colon cancer has emerged as one of the deadliest of its kind contributing to a high mortality rate. This starts with the formation of a pouch-like structure called polyp and early detection of this polyp can help in reducing the mortality rate significantly. Detecting polyps manually is a hectic and erroneous process as some polyps are small and look like their surroundings. Deep learning-based automatic detection of polyps can help this cause and make the process time-efficient using various segmentation models. Traditional segmentation lacks the capability of detecting multiple polyps in a single area as this shows gross area affected only. To overcome these lacking we proposed a deep learning-based approach combining a modified U-Net model and Watershed algorithm to segment the colonoscopy images. This produces mask files in a colored format where the boundaries among multiple polyps are well-defined and indicated by different colors. This helps in diagnosing a patient well and reduces the miss rate for polyps. Our proposed work has been trained on the Kvasir-SEG dataset and implemented on Kvasir V2 polyp class data for segmentation. Our combination of U-Net and Watershed algorithm returned an accuracy of 99.24% and 94.83% for training and validation, respectively. Our primary target is to detect, segment, and determine the size and number of polyps with precision to automate this whole process which is backed by the accuracy and other metrics obtained from the experiment.