Multiplying Bayes Factors Across Studies: Which Hypotheses Are We Testing?

Bayesian evidence synthesis (BES) is a method for evaluating the empirical support for a common theory across multiple studies. In each study, the theory predicts a set of order constraints on the parameters of interest. Evidence is aggregated by multiplying the Bayes factors of the individual studies, yielding the product Bayes factor (PBF). The numerator of the PBF corresponds to the joint predicted hypothesis that the theory holds across different contexts, implying that the predicted order constraints are correct in each study. Since the Bayes factor is a relative measure of evidence, interpreting the PBF requires a clear understanding of the alternative hypothesis. The joint alternative is represented by the denominator of the PBF and states that the respective study-specific alternative hypothesis holds in each study. However, this joint alternative may lack both a clear substantive interpretation and a theoretical justification. We investigate how different combinations of study-specific alternative hypotheses give rise to distinct joint alternatives, focusing on three alternatives common in order-constrained inference: the null, the unconstrained, and the complement hypothesis. In particular, we show that multiplying Bayes factors involving study-specific complements does not test the theory against its complement (i.e., the hypothesis that the theoretical predictions are false in at least one study). Finally, we provide recommendations for specifying meaningful joint alternative hypotheses that support a clear interpretation of BES results and are useful for theory testing.