Nuclei Segmentation Using Multiheaded U-Net and Shearlet-Based Unsharp Masking

An automated nuclei segmentation is an important technique for understanding and analyzing cellular characteristics that eases computer-aided digital pathology and is useful for disease diagnosis. However, this task is difficult because of the diversity in nuclei size, blurry boundaries, and several imaging modalities. A convolutional neural network (CNN) based Multi-headed U-Net (M-UNet) framework has been proposed to address such issues. This architecture uses filters of different kernel sizes for multiple heads to extract multi-resolution features of an image. Shearlet based unsharp masking (SBUM) method is proposed for pre-processing which primarily emphasizes features like contours, boundaries, and minute details of the source image. In this paper, a hybrid loss function is formulated which includes intersection over union (IOU) loss and Dice loss along with binary cross entropy loss. The hybrid loss function is tried to be minimized by the optimization algorithm, and the higher metrics values during the testing phase represent better segmentation performance in the spatial domain. The proposed method yields superior segmentation images and quantitative findings as compared to the state-of-the-art nuclei segmentation techniques. The proposed technique attains IOU, F1Score, Accuracy, and Precision values of 0.8325, 0.9086, 0.9651, and 0.9001 respectively.