Revolutionizing Colon Histopathology Glandular Segmentation Using an Ensemble Network With Watershed Algorithm

ABSTRACT Colorectal adenocarcinoma, the most prevalent form of colon cancer, originates in the glandular structures of the intestines, presenting histopathological abnormalities in affected tissues. Accurate gland segmentation is crucial for identifying these potentially fatal abnormalities. While recent methodologies have shown success in segmenting glands in benign tissues, their efficacy diminishes when applied to malignant tissue segmentation. This study aims to develop a robust learning algorithm using a convolutional neural network (CNN) to segment glandular structures in colon histology images. The methodology employs a CNN based on the U‐Net architecture, augmented by a weighted ensemble network that integrates DenseNet 169, Inception V3, and Efficientnet B3 as backbone models. Additionally, the segmented gland boundaries are refined using the watershed algorithm. Evaluation on the Warwick‐QU dataset demonstrates promising results for the ensemble model, by achieving an F1 score of 0.928 and 0.913, object dice coefficient of 0.923 and 0.911, and Hausdorff distances of 38.97 and 33.76 on test sets A and B, respectively. These results are compared with outcomes from the GlaS challenge (MICCAI 2015) and existing research findings. Furthermore, our model is validated with a publicly available dataset named LC25000, and visual inspection reveals promising results, further validating the efficacy of our approach. The proposed ensemble methodology underscores the advantages of amalgamating diverse models, highlighting the potential of ensemble techniques to enhance segmentation tasks beyond individual model capabilities.