Order Effects in Evidence Chains: Normative and Naïve Evaluations

This is a first exploration into a newly identified reasoning error. We explore normative (derived from Bayes' rule) and naïve (empirical data) evaluations of how order of reliability within chains of evidence (e.g., hearsay testimony) impacts overall evidential value. In a novel paradigm, we swap the position of two witnesses within the chain to determine the effect of order, when these witnesses differ in their reliability. First, a probabilistic (Bayesian) assessment is provided, including both quantitative and qualitative explanations. Second, lay reasoner qualitative intuitions are measured, using Bayesian predictions as a benchmark for accuracy. Lay reasoners significantly deviate from Bayesian predictions. Three quarters (75.41%) made an error when evaluating order effects in hearsay testimony, with 49.18% wrongly concluding that order has no impact. Only 24.59% correctly judged that the preferential order had greatest evidential value. We illustrate how hearsay testimony is inherently complex and an optimal evaluation is nonintuitive.