Stability-guided hyper-parameter tuning for stability selection

Stability selection is a widely used resampling-based framework for variable selection in high-dimensional settings. This paper leverages an established stability estimator to quantify the overall stability of stability selection outcomes and to identify a Pareto-optimal regularization parameter, denoted by λstable, defined as the smallest regularization value achieving a pre-specified level of selection stability. Fixing this regularization value enables principled calibration of key stability selection hyper-parameters, including the selection threshold and the expected number of false selections, while respecting existing theoretical upper bounds. Moreover, analysis of the convergence of stability estimates across successive sub-samples provides guidance on the number of sub-samples required, addressing a notable gap in prior work. Numerical experiments on synthetic and real bioinformatics datasets demonstrate that the proposed approach identifies regularization values that achieve substantially improved selection stability with only minor loss in predictive accuracy. The results also show that highly correlated data can lead to unstable stability selection outcomes, highlighting the importance of evaluating selection stability before interpreting selected variables. An accompanying R package, stabplot, is introduced to support practical implementation of the proposed methodology.