Implications of assuming common within-source distributions and their effect on evidence interpretation

Within the forensic science community, there is a need for a statistically rigorous determination of whether an exclusionary difference exists; this determination is integral to the Kirkian Two-Stage approach to evidence interpretation (Parker 1966). If the known source is not excluded as the source of the questioned object in the first stage, then the examiner must determine the rate at which competing alternative sources are excluded. Current methods are typically constructed to ensure the same false exclusion rate for each source. For example, in ASTM glass standards E2927-16e1 and E2330-19, an exclusionary difference occurs if any of the standardized differences between the measured trace element concentrations is greater than a fixed threshold of four. However, if the algorithm’s score function has a distribution that varies by source, then the corresponding thresholds will need to vary as well. In this work, we review strategies for identifying when the within-source distribution of scores varies between sources; methods for estimating thresholds; remedial approaches such as pooling subsets of sources together; and implications of this type of variability among the sources to the Kirkian and likelihood ratio (LR) approaches. We illustrate these methods with example data from traces such as glass and improvised explosive device components. Although the focus is on the Two-Stage approach, this work is also important for LR-based methods due to the need to estimate a likelihood function from just a few observations from a specified source. The discussed remedial methods also apply to the LR paradigm for evidence interpretation.