Segment Any Nuclei: A Prompt-free Segment Anything Model for Nuclei from Histology Image

Accurate cell nuclei segmentation is crucial for characterizing cell morphology and elucidating disease types in medical image analysis. However, fully-supervised segmentation models often have limited generalization to unseen classes or domains due to being trained on specific modalities or cell types. The recent Segment Anything Model (SAM) has shown promise for interactive instance segmentation and zero-shot generalization. However, its effectiveness is hindered by the sparse and distributed nature of nuclei. It also requires a large number of user prompts that scale with the number of nuclei in the image. To overcome these challenges, we introduce Segment Any Nuclei (SAN), a novel foundation model tailored specifically for nuclei segmentation. SAN is trained on an extensive multi-modal dataset containing diverse nuclei instances from various imaging modalities, staining methods, and tissue types. Unlike previous SAM approaches that rely on manual prompts, SAN incorporates an innovative auto-prompting auxiliary segmentation network, which enables the model to make predictions without manual intervention while still allowing for manual interaction when needed. We evaluate SAN on a large dataset of 9,244 images and demonstrate state-of-the-art nuclei segmentation performance, surpassing both fully-supervised approaches and other SAM-like models. SAN represents a step towards more generalizable, efficient and interactive nuclei segmentation. Our code is available at https://github.com/Mrliujunwen/SAN.