Beyond “non-significant” results: Why and how to test for practical equivalence

≥ 0.05 as showing "no effect" or "no difference" remains one of the most widespread and problematic misinterpretations in the scientific literature. A statistically nonsignificant result shows only that the data do not provide strong evidence for a difference. This distinction matters because such findings can arise for two very different reasons: Either there is no meaningful difference, or a meaningful difference is present but cannot be detected reliably because of limited sample size or high variability. We highlight equivalence testing as a practical framework for distinguishing between these possibilities. Using the two one-sided tests (TOST) approach, investigators can formally test whether effects large enough to be scientifically, clinically, or practically meaningful can be ruled out. We explain the logic of TOST, show how it distinguishes practical equivalence from inconclusive evidence, and illustrate its use in paired and unpaired comparisons. To support broader adoption, we also introduce a freely available online calculator that enables researchers to perform common equivalence tests without coding.

Authors

Institutions

Publication Details

Journal
Proceedings of the National Academy of Sciences
Published
2026-08-14
DOI
https://doi.org/10.1073/pnas.2611548123
Primary Topic
Meta-analysis and systematic reviews
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Beyond “non-significant” results: Why and how to test for practical equivalence

Jakub Tomek, David Eisner, Aaron Caldwell
Proceedings of the National Academy of Sciences
Meta-analysis and systematic reviews
article

Beyond “non-significant” results: Why and how to test for practical equivalence

Jakub Tomek, David Eisner, Aaron Caldwell
article en

Abstract

≥ 0.05 as showing "no effect" or "no difference" remains one of the most widespread and problematic misinterpretations in the scientific literature. A statistically nonsignificant result shows only that the data do not provide strong evidence for a difference. This distinction matters because such findings can arise for two very different reasons: Either there is no meaningful difference, or a meaningful difference is present but cannot be detected reliably because of limited sample size or high variability. We highlight equivalence testing as a practical framework for distinguishing between these possibilities. Using the two one-sided tests (TOST) approach, investigators can formally test whether effects large enough to be scientifically, clinically, or practically meaningful can be ruled out. We explain the logic of TOST, show how it distinguishes practical equivalence from inconclusive evidence, and illustrate its use in paired and unpaired comparisons. To support broader adoption, we also introduce a freely available online calculator that enables researchers to perform common equivalence tests without coding.

Proceedings of the National Academy of SciencesVol. 123(33)
University of Manchester (GB), University of Oxford (GB), University of Arkansas for Medical Sciences (US)
Openalex Percentile: Top 7%
Meta-analysis and systematic reviews
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.