Risk-Utility Trade-offs in Selective Decision Gates: A Synthetic Study of Outlier Contamination
A selective decision gate can reduce wrong decisions by withholding actions while losing much of the baseline system's useful output. We examine this trade-off in a classical, four-hypothesis sequential simulator through four archived experimental stages: an exploratory pilot, a prospectively specified robustness replication, a paired factorial diagnosis, and an exploratory feature extension. Each stage uses 64 trials with separate training and calibration data; each matched evaluation contains 256 blocks of four episodes per trial. A ridge logistic scorer ranks baseline decisions, and a conformal risk control gate targets a 5% rate of wrong authorizations across all episodes. In the revised generator, the seven-feature gate attains 4.65% marginal risk but retains only 74.30% of baseline-correct decisions, failing its 80% requirement. The factorial diagnosis associates outlier contamination with a 19.97 percentage-point retention loss, compared with 3.41 points for a dependence bundle. Adding three confidence-path shock features improves retention from 73.61% to 81.03% in the outlier cell, with a paired trial gain of 7.42 points (95% bootstrap interval: 6.75 to 8.06). Nevertheless, standard-profile retention remains below 90%, and combined-condition retention reaches only 79.81%. The original STOP and subsequent PIVOT decisions therefore remain justified. The contribution is a reproducible failure analysis and partial repair in a specified synthetic setting, rather than a new risk-control theorem or evidence of deployment safety.
Authors
- Mariusz Kulma (ORCID: https://orcid.org/0009-0000-5550-8723)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-13
- DOI
- https://doi.org/10.5281/zenodo.22731540
- Primary Topic
- Formal Methods in Verification
- Type
- preprint