Operating characteristics of Bayes factors

Abstract Bayes factors are Bayesian test statistics whose numerical post-experimental values quantify the evidence provided by an experiment in favour of a statistical hypothesis with respect to an alternative hypothesis. Their use in clinical trials has been often advocated, conditional to explicit control of their frequentist operating characteristics, namely type-one error and power. In this article we revisit and further explore the formal relationships between Bayes factors and standard optimal frequentist testing procedures for both one-sided and two-sided hypotheses on a normal mean, frameworks that are widely used in experimental contexts. With regard to tests based on Bayes factors, these relationships allow one: (i) to prove frequentist optimality and to highlight differences between the one-sided and the two-sided cases; (ii) to obtain explicit formulas to compute type-one error and power from those of the related frequentist test; and (iii) to determine the expression of the so-called probability of success from that of the frequentist test. We illustrate the main ideas in several numerical examples and in an application on the use of mirtazapine in dementia patients, where the sample size determination problem is also considered.