Selective Foundation-Model Intervention for Vertebral CT Segmentationunder Predictive Uncertainty

Abstract Foundation segmentation models offer strong generalization capabilities, but their intervention is not necessarily beneficial when a task-specific model already provides reliable predictions. We propose an uncertainty-guided selective collaboration framework (UGSCF) for vertebral CT segmentation, in which the Segment Anything Model (SAM) acts as an on-demand correction proposer rather than a mandatory second-stage segmenter. UGSCF follows a route–propose–verify–correct strategy: a task-specific UNet first generates the default segmentation and predictive uncertainty; uncertainty determines where SAM intervention should be allocated; SAM-derived proposals are restricted to uncertain disagreement regions; and a candidate-value verifier determines whether a proposed correction should modify the original prediction. Experiments on VerSe19 showed that unconditional SAM intervention was frequently harmful, with direct SAM output degrading 91.66% of eligible validation slices. Predictive uncertainty nevertheless provided useful allocation information, with the fraction of beneficial interventions increasing from 2.04% in the lowest-uncertainty quartile to 30.49% in the highest. Under nested subject-level validation, the best deployable candidate-value policy achieved a mean volumetric Dice of 0.814658, compared with 0.814139 for UNet alone. SAM was invoked for 19.53% of foreground-positive validation slices, while only 9.49% received an accepted correction, of which 65.62% were beneficial. The validation gain was small and not statistically supported (95% CI: −0.000312 to 0.001493; p = 0.307821). In a post-hoc 40-subject test-set evaluation, UGSCF achieved a mean volumetric Dice of 0.825360 compared with 0.825776 for UNet (∆ = −0.000416; p = 0.265373). These results indicate that collaboration between task-specific and foundation segmentation models is better formulated as a selective decision process than as unconditional model fusion. Rather than establishing a consistent improvement in aggregate segmentation accuracy, UGSCF provides a hierarchical mechanism that separates where additional foundation-model computation is allocated from whether the resulting proposal is trusted.