The Threshold, Not the Algorithm: Decomposing the Monetary Value of Model Selection versus Decision-Threshold Calibration in Credit Scoring

Cost-sensitive re-evaluations of credit-scoring models typically report savings as a percentage of exposure, a form that is scientifically sound but difficult for a risk or finance team to act on directly. This paper converts the same class of evaluation into currency and asks a more operational question: of the total money a lender leaves on the table, how much is recoverable by choosing a better algorithm, and how much by simply setting a better decision threshold. Using a paired, Optuna-tuned, repeated stratified cross-validation re-evaluation of four classifiers (Logistic Regression, Random Forest, XGBoost, LightGBM) on the UCI Default of Credit Card Clients benchmark (6,000-account holdout, NT$1.01 billion in credit exposure, all figures in New Taiwan Dollar, NT$, the currency of the source dataset), current practice - selecting the AUC-best model (LightGBM) and applying the conventional 0.5 decision threshold - carries an expected loss of NT$99.48 million. A cost-aware alternative - selecting the cost-best model (Random Forest) at a nested, cost-optimized threshold - carries an expected loss of NT$12.16 million: a saving of NT$87.32 million (87.8%), or NT$14,554 per account. A two-path, order-independent decomposition of this saving attributes 99.83% of it to threshold calibration alone and just 0.17% to the choice of algorithm - an issuer that only recalibrates its decision threshold, without ever touching its model, captures essentially all of the available saving, a result consistent with our finding that the four models are not statistically distinguishable in cost terms once each is given its own optimal threshold. We illustrate the scale of this effect for a hypothetical 100,000-account portfolio (linear extrapolation, NT$1.46 billion in projected savings) and discuss the implication for how risk and data-science teams should prioritize engineering effort: threshold governance, not model replacement, is where the money is.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.24304567
Primary Topic
Financial Distress and Bankruptcy Prediction
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

The Threshold, Not the Algorithm: Decomposing the Monetary Value of Model Selection versus Decision-Threshold Calibration in Credit Scoring

Nezar Abdilah Prakasa
2 links
Zenodo (CERN European Organization for Nuclear Research)
Financial Distress and Bankruptcy Prediction
preprint

The Threshold, Not the Algorithm: Decomposing the Monetary Value of Model Selection versus Decision-Threshold Calibration in Credit Scoring

Nezar Abdilah Prakasa
preprint en

Abstract

Cost-sensitive re-evaluations of credit-scoring models typically report savings as a percentage of exposure, a form that is scientifically sound but difficult for a risk or finance team to act on directly. This paper converts the same class of evaluation into currency and asks a more operational question: of the total money a lender leaves on the table, how much is recoverable by choosing a better algorithm, and how much by simply setting a better decision threshold. Using a paired, Optuna-tuned, repeated stratified cross-validation re-evaluation of four classifiers (Logistic Regression, Random Forest, XGBoost, LightGBM) on the UCI Default of Credit Card Clients benchmark (6,000-account holdout, NT$1.01 billion in credit exposure, all figures in New Taiwan Dollar, NT$, the currency of the source dataset), current practice - selecting the AUC-best model (LightGBM) and applying the conventional 0.5 decision threshold - carries an expected loss of NT$99.48 million. A cost-aware alternative - selecting the cost-best model (Random Forest) at a nested, cost-optimized threshold - carries an expected loss of NT$12.16 million: a saving of NT$87.32 million (87.8%), or NT$14,554 per account. A two-path, order-independent decomposition of this saving attributes 99.83% of it to threshold calibration alone and just 0.17% to the choice of algorithm - an issuer that only recalibrates its decision threshold, without ever touching its model, captures essentially all of the available saving, a result consistent with our finding that the four models are not statistically distinguishable in cost terms once each is given its own optimal threshold. We illustrate the scale of this effect for a hypothetical 100,000-account portfolio (linear extrapolation, NT$1.46 billion in projected savings) and discuss the implication for how risk and data-science teams should prioritize engineering effort: threshold governance, not model replacement, is where the money is.

Zenodo (CERN European Organization for Nuclear Research)
Financial Distress and Bankruptcy Prediction
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon