Separation-like irregularity and sample size optimism in high-discrimination logistic prediction models

Closed-form minimum sample size criteria for developing logistic prediction models, such as the Riley framework implemented in pmsampsize , are widely used but may become optimistic when anticipated discrimination is high. We conducted a Monte Carlo simulation study to compare the formula-based recommended development sample size, n R i l e y , with an empirical required sample size, n r e q , defined by out-of-sample calibration-slope stability under repeated development sampling. Scenarios fixed the candidate parameter dimension at p = 10 and crossed predictor distribution (normal, standardized skewed continuous, binary), signal density (dense versus sparse), prevalence ( ϕ ∈ { 0.05 , 0.10 , 0.20 } ), and target discrimination ( A U C t a r g e t ∈ { 0.70 , 0.75 , 0.80 , 0.85 , 0.90 } ), with intercept and signal strength calibrated to match targets. We defined n r e q as the smallest n such that 𝔼 ( b n ) ≥ 0.90 and Pr ( b n &