Can You Trust That Forecast? A Local Diagnosis and Reshaping of Predictive Distributions

Probabilistic forecasts describe not just what is expected to happen but how uncertain that expectation is — yet a forecast that is well calibrated on average can still be unreliable under specific conditions. This article gives an accessible introduction to checking and correcting predictive distributions locally, using Local Amortized Diagnostics and Reshaping (LADaR; Dey et al., 2025). We explain the probability integral transform (PIT) and how local PIT-CDF curves and local discrepancy scores reveal where and how a model's uncertainty is wrong, then show how LADaR reshapes the predictive distribution only where correction is needed. The method is illustrated with a synthetic example where the true distribution is known, and with a real forecasting problem: predicting Pittsburgh's daily maximum temperature, where the base model's uncertainty is least reliable during extreme heat events such as the June 2024 Heat Dome and June 2025 heat wave. Examples were developed for a Carnegie Mellon University undergraduate statistics lab.