Integrating SAM priors with U-Net for enhanced multiclass cell detection in digital pathology

In digital pathology, the accurate detection, segmentation, and classification of cells are pivotal for precise pathological diagnosis. Traditionally, pathologists manually segment cells from pathological images to facilitate diagnosis based on these results and other critical indicators. However, this manual approach is not only time-consuming but also prone to subjective biases, which significantly hampers its efficiency and consistency in large-scale applications. While classic segmentation networks like U-Net have gained widespread adoption in medical imaging, their integration with external prior features remains limited, thereby restricting the potential enhancement of segmentation accuracy. Although the large model SAM, renowned for its capability to "segment anything", has shown promise, its application in the specialized field of medical image processing presents considerable challenges. Direct application of SAM to medical scenarios often fails to yield optimal results. To overcome these limitations, this paper proposes a novel prior-guided joint attention mechanism. This method effectively integrates the prior features generated by SAM with the foundational features extracted by U-Net, achieving superior cell segmentation and classification. Extensive experiments on public datasets reveal that the proposed method significantly surpasses both standalone U-Net and approaches that merely augment inputs by overlaying prior features onto color channels. This advancement not only enhances the utility of large models in medical applications but also lays the groundwork for the evolution of intelligent pathological diagnostic technologies.