Advancement in semantic segmentation techniques: a comprehensive review for semantic segmentation of colorectal polyps using deep learning

Abstract Accurate segmentation of medical diseases, particularly in the detection and delineation of colorectal polyps, remains a critical challenge in medical diagnostics, as traditional image processing techniques often fail to capture the complexity and variability of polyp data, leading to inconsistent results and potentially impacting clinical outcomes. This review aims to study and analyze the latest 110 deep learning (DL) techniques from 2018 to 2024 with more than 100 open-source codes for polyp segmentation in a single review paper, with a focus on semantic networks, attention mechanisms, multiscale cascades, and transformer architectures, exploring their potential to improve the accuracy and robustness of colorectal polyp segmentation. Through a comprehensive review of existing literature, we classify and assess key methodologies, including single network models, multiple network models, hybrid models, and transformer-based models, particularly in their ability to handle variability in polyps’ patterns and enhance model interpretability. Our findings indicate that transformer-based architectures, especially those employing self-attention mechanisms, significantly enhance segmentation accuracy compared to traditional convolutional approaches, while semantic networks and multiscale cascades also show improved performance in addressing polyp variability across different scales. However, these advanced models bring challenges in terms of computational complexity and resource demands. The integration of these DL techniques offers transformative potential for improving diagnostic accuracy in colorectal polyp segmentation, and future research should focus on optimizing these models for clinical application by addressing computational demands and enhancing generalizability across diverse datasets, providing a roadmap for future development in colonoscopy imaging.