Polyp Segmentation with Transformer-CNN Integration and Multi-Scale Feature Fusion

Polyp segmentation is, thus far, an essential task in the medical domain, especially in diagnosing and treating colorectal diseases. Polyps are abnormal tissue developments, which emerge on the inner lining of the colon, and some types have the potential to progress to colorectal cancer if not detected and treated promptly. Polyps are variable in shape and surface texture. Because the quality of polyp images relies on endoscopic protocols, elements like lighting conditions, endoscopic equipment, and the polyps' surface characteristics can be detrimental to image quality. Recently, Convolutional Neural Networks (CNNs) have appeared as an effective method for handling local features; however, due to the limitations regarding long-range dependencies and challenges in capturing global features, pure CNN models often fail to achieve high accuracy on complex datasets such as those for polyp segmentation. Additionally, Transformer architectures with attention mechanisms have become advanced approaches because of their robustness in addressing global contexts; still, they require considerable computational resources. To improve accuracy in polyp segmentation, we propose an efficiency-focused U-shape model, employing PVTv2 pre-trained as the backbone, coupled with a child block incorporating Squeeze-and-Excitation Attention Block (SEAB). Furthermore, we present a novel feature fusion module that effectively combines multi-scale features, boosting the model's performance. Our experiments with Kvasir-SEG and CVC-ClinicDB demonstrate that PMFNet provides improved accuracy in comparison with other state-of-the-art models. The code will be available at: https://github.com/lamtt262004/PMFNet.