Quantifying Observation Representativeness Errors Using a Spatial Verification Method: A Lightning-Based Illustration

Abstract Accounting for representative mismatches helps reduce ambiguity when interpreting verification results. For example, there can be a difference in the spatial extent between the physical phenomenon and its data instance. In this work, we use lightning to illustrate this. Lightning is near instantaneous in time, highly localized, spatially sparse yet three-dimensional and often processed into a two-dimensional point location with a time stamp. Numerical weather prediction (NWP) forecasts of lightning flash origin density are typically derived from parameterized subgrid-scale processes. The minimum areal extent of any forecast quantity is dictated by the model grid length, such that smaller-scale phenomena are inflated (and thus misrepresented) in terms of scale. The model representation of reality is often too large, and the observation instance to compare to is too small. Gaussian kernel dressing is one way of increasing the footprint of highly localized observations to help mitigate against the representativeness mismatch when using such observations for evaluating model output, even in gridded form. When adjusting the observation “footprint,” the observation characteristics must be examined and used as a guide. The coverage–distance–intensity (CDI) method is used to examine what impact spatial representativeness mismatches have on the interpretation of spatial verification method output. We found a 38% change in the median coverage component between using dressed and undressed observation fields, demonstrating that the spatial method can quantify (and is affected by) representativeness mismatches. While the study is lightning based, the study attempts to synthesize some general recommendations for mitigating against representativeness error. Significance Statement The purpose of this study was to practically demonstrate the effect that any spatial mismatches between what a model forecast provides and what the observation represents has on verification results, even for a spatial verification method. The results show that mitigating for the mismatch leads to a large change in one of the components of the spatial verification method, with the implication that if this kind of mismatch is not addressed, the wrong conclusions could be drawn from the results. Furthermore, the study suggests that the spatial verification method could be used more widely to test for and quantify whether any mitigation for a spatial mismatch is successful.