A methodology for nuclei segmentation in H&E images using EfficientNetB7 guided U-Net and multiple data augmentation strategies

Automating nuclei segmentation in Hematoxylin and Eosin (H&E) stained images is crucial for advancing computational pathology. Despite significant research, challenges such as overlapping nuclei, varying scanner resolutions, and diverse nuclear morphologies continue to hinder segmentation accuracy. In this paper, we propose a deep-learning based methodology that integrates multiple data augmentation strategies with a U-Net architecture enhanced by the EfficientNetB7 encoder. To enhance generalization, we train the model using a combined dataset from MoNuSeg2018, CPM-17, and CoNSeP, exposing it to diverse staining techniques and tissue types. We then evaluate its robustness on the unseen CryoNuSeg dataset, which consists of fully annotated frozen H&E stained histological images from 10 human organs, as well as on the test sets of MoNuSeg2018, CPM-17, and CoNSeP. The proposed methodology achieves competitive Aggregated Jaccard Index (AJI) scores of 0.8042, 0.7633, 0.7068, and 0.7118 on MoNuSeg2018, CPM-17, CoNSeP, and CryoNuSeg, respectively, demonstrating its effectiveness in segmenting nuclei across diverse histopathological images.