Confidently Uncertain: Validating Satellite ECV Measurement Uncertainty Estimates

The importance of uncertainty estimates for Essential Climate Variable (ECV) data records is well recognized. Most ECV observing systems now estimate and report uncertainties as part of the measurement and retrieval procedure instead of leaving uncertainty characterization to a diagnostic "ex-post" validation/evaluation procedure. This paper focuses on the validation or evaluation of these prognostic “ex-ante" uncertainties provided with satellite ECV climate data records. In recent years, established validation protocols (primarily focused on the validation of the measurements themselves) have been extended to provide feedback on the prognostic uncertainties. Moreover, dedicated methods for uncertainty validation have been developed and applied across the Earth observation community. In this review, we classify, describe, and illustrate these approaches under three categories: (1) those relying on comparison to independent in-situ or other ground-based reference data, (2) those exploiting self-co-locations, i.e., the comparison between consecutive measurements by the same instrument under appropriate conditions, and (3) those integrating a multitude of correlative (satellite) data sets. The principles behind each method are described with a common nomenclature, facilitating a clear identification of strengths and weaknesses, and the illustrations are chosen to cover the three Global Climate Observing System (GCOS) ECV domains (atmosphere, ocean, terrestrial) toward the aim of facilitating cross-domain knowledge transfer. Finally, the most common and critical gaps and challenges are listed to support the scoping of future work.