Statistical tests should inform behaviour in theory and practice [Author Accepted Manuscript]

Objectives: Statistical hypothesis testing cannot establish a hypothesis as certainly — or even approximately — true, but it is valuable in guiding behavior as if it were true. To make decisions in theory and practice, statistical testing in psychology should weigh the Type I (α) and Type II (β) error probabilities according to their respective costs. Methods: We outline costs and how to approach the α-β balance for six common decision-making questions: (1) Should a hypothesis be considered tentatively accepted? (2) Should a model or theory cover a newly proposed effect? (3) Should a new hypothesis be proposed? (4) Should a new intervention be introduced? (5) Are the psychometric properties of an instrument sufficient to use it? (6) Are the assumptions of a statistical method met so that it may be used? We give a worked example with real cost figures for question (5). Results: We argue that parsimony — prioritizing low α over low β — often serves as a useful default stance in questions (1) to (5). This resembles the general preference for focusing on fewer, larger and more robust effects to allocate resources efficiently and to make scientific communication more reliable. Conclusions: The parsimony default invites transparent counterarguments to override it, justifying a preference for small β. Examples include adding promising intervention components with little risk of adverse effects to well-established interventions. As with (6), statistical tests can sometimes introduce unnecessary errors when a sound decision can be made on substantive grounds alone, for example when setting up a model to test a given hypothesis.