Sharper constraints without overconfidence: a scoring rule for perfect-model tests and climate projection weighting

Abstract. Projections of future climate carry large uncertainties, commonly quantified through the spread of a multi-model ensemble. The models contributing to such ensembles, however, are neither equally plausible nor independent. Weighting schemes address this by rewarding models that better reproduce the observed climate and penalizing redundant models, with both contributions computed from distances in a chosen predictor space. The performance weighting is governed by a free parameter, calibrated through a perfect-model test in which each model is treated in turn as a pseudo-reality. This parameter is set by following the philosophy of "constraining as much as possible without being overconfident'', monitoring how often the perfect models fall inside a pre-chosen confidence interval. Two problems follow: the test is not sensitive to the position of the pseudo-truth within the interval, and it assumes that some value of the performance parameter hits the target frequency, which is not guaranteed. Here, we reformulate perfect-model tests using the continuous ranked probability score (CRPS), a statistic that compares the entire weighted distribution with the pseudo-reality, and we select the performance parameter at its minimum. The use of this metric shifts the objective of weighting from constraining the projections to predicting well, with the narrowing of the ranges following as a consequence. On synthetic data, the resulting scheme constrains more than its coverage-based counterpart without suffering from overconfidence, provided that the model families are conservatively specified. We apply the two criteria to CMIP6 precipitation projections over land under SSP5-8.5, comparing predictor distances based on sea-level pressure fields and causal networks inferred from sea-level pressure time series. The CRPS-based weights narrow the very likely range by up to 32% for 2041-2060, while the end-of-century projections remain weakly constrained. Causal-network distances carry usable information and yield appreciable spread reductions, but they rest on shallower optima than a simple area-weighted RMSE, which remains the most reliable predictor distance we tested.