To What Are We Inferring? The Widespread Misuse of Inferential Testing in the Most Cited Criminology and Criminal Justice Journals

Null Hypothesis Significance Testing (NHST) is widely used in criminology and criminal justice journals. Where either random selection of a sample or random assignment to treatment and control groups is absent, the meaning of inferential testing becomes unclear. Using a sample of articles published in 18 of the top ranked (h5-index) journals in criminology, criminal law, and policing, we examine (1) how often authors use NHST after violating the assumption of random selection/assignment, (2) whether authors focus more on statistical rather than substantive importance, and (3) whether this leads authors to ignore large statistically non-significant effects based on a flawed test. Our results suggest that it is commonplace to apply statistical procedures and interpret results with little attention to how data were generated and how results should be assessed. We recommend the use of exploratory research methods, better statistical training for students, and addressing publishing standards in our field.