PAC-Net: patch adaptive cut-off network with differentiable module-wise K-learning for robust and efficient medical image segmentation

OBJECTIVE: Existing sparse attention networks in medical image segmentation suffer from fixed K values across modules and scales, hindering effective adaptation to the wide variability in lesion sizes and shapes. The objective of this study is to develop an adaptive, differentiable sparse attention mechanism to solve this issue for both the robustness and the segmentation accuracy of medical image analysis systems. APPROACH: We propose PACNet (Patch Adaptive Cut-off Network), which embeds an Entropy-Guided Differentiable K-Selection (EGDK) module to learn data-dependent sparsity ratios. EGDK combines Gaussian Soft Indexing with a Straight-Through Estimator (STE) to preserve end-to-end differentiability during optimization, addressing the gradient blocking issue in conventional discrete Top-K selection. PACNet is evaluated on eight datasets spanning five imaging modalities under a unified training and evaluation protocol. MAIN RESULTS: PACNet achieves superior or highly competitive performance against state-of-the-art methods, obtaining the best average DSC of 90.57% across the eight datasets. It improves average DSC by +1.79% over BRAU-Net++ (which also uses Top-K-style sparse selection). In addition, PACNet achieves strong efficiency with 6.82M parameters and 6.02G FLOPs. SIGNIFICANCE: The results show that adaptive and differentiable K selection is more suitable than fixed or discrete sparse selection for medical image segmentation. PACNet effectively suppresses background clutter and preserves fine-grained anatomical details, providing a practical and clinically meaningful solution for robust general medical image segmentation.