EMSSD: Two-Stage Model Enhancing Medical Image Segmentation Based on Stable Diffusion

Denoising Diffusion Probabilistic Models have recently demonstrated remarkable in various image generation tasks. With robust nonlinear modeling capacity and superior generalization performance, denoising diffusion models are being progressively applied in medical image segmentation. However, popular diffusion models trained with 3D patch-based or 2D slice-wise inputs may lack a global anatomical perception of medical tissues and performing operations in pixel space incurs substantial computational overhead. In this study, we propose a two-stage framework, called Enhancing Medical image Segmentation based on Stable Diffusion (EMSSD), to elevate segmentation accuracy. A 3D segmentation model is pretrained in the first stage to obtain coarse segmentation maps which along with the input images are jointly encoded as constraints into latent diffusion model to direct model's attention to the target region in the second stage. To sharpen the guidance of fuzzy boundary prior from coarse segmentation results while adequately leveraging the medical images, we design a Condition Fusion Module (CFM) to aggregate both conditions. We also introduce a single-step reverse process to replace multi-step to optimize time efficiency. Experiments conducted on two datasets have substantiated the effective-ness and generalization of our approach.