Platt Scaling Versus Isotonic Calibration: Brier Scores on Five UCI Classifiers and Why Recalibration Can Raise the Score

Probabilistic classifiers are often recalibrated by Platt scaling (logistic regression on scores) or isotonic regression. On five UCI binary data sets, logistic regression probabilities are evaluated out of fold, with calibrators fit on a held-out slice of each training fold. Mean Brier scores across the five data sets are 0.0746 (raw), 0.0779 (Platt) and 0.0817 (isotonic): recalibration raises the cross-data-set average Brier score. Banknote and spambase improve under isotonic (Brier falls by 0.0034 and 0.0020), while wdbc, ionosphere and sonar worsen. Platt beats isotonic on average (0.0779 against 0.0817). The Murphy reliability-resolution-uncertainty split with ten equal-frequency bins tracks the ranking of methods but is only approximate under coarse binning. Recalibration is not guaranteed to help when the base model is already well calibrated or when the calibration slice is small.