NuDetect: A point annotation-based framework for nuclei detection using density estimation and conformal thresholding

Whole Slide Imaging (WSI) generates vast data sets in histopathology. Manual annotation is impractical and time consuming. There is, thus, a dire need for effective analysis tools. However, a lack of annotated data hampers supervised learning of models that generalize well across domains. Point annotations have emerged as a practical remedy. Motivated by the fact that the randomness of the tissue slice angle and depth renders size measurements of nuclei—such as it would be provided by segmentation—meaningless (unlike in other medical tasks), point annotations are efficient and useful due to their sparseness. In this paper, we formulate the task of nuclei detection as a density estimation problem. We use a U-Net architecture with PoolFormer encoders as the basis to compute point-annotations for nuclei detection. Specifically, we use Gaussian kernels to generate target density masks from a segmented data set and use isocontouring to separate overlapping nuclei. We show that conformal prediction can compute a near-optimal threshold for contouring. This significantly enhances our detection rate. To address cross-domain generalization issues, our framework uses color normalization. As a result, our framework sets a new state-of-the-art in nucleus localization on both the PanNuke and MoNuSeg data sets, and we demonstrate our cross-domain generalization capabilities using samples of the TCGA data set. • We propose nuclei detection as a density estimation task using point annotations. • A U-Net with PoolFormer encoders generates density maps from sparse labels. • Isocontouring and conformal prediction enable accurate separation of nuclei. • Color normalization improves cross-domain generalization in histopathology. • Our method sets a new state-of-the-art on PanNuke and MoNuSeg datasets.