Do Bayesian Intervals Fail on the Same Side? Noninformative Priors, Skewed Likelihoods and the Direction of Small-Sample Interval Failure for a Difference in Means

Background: When small-sample intervals for a difference of two arithmetic means fail under skewness, they fail mostly on one side, given by the sign of the skewness balance . Bayesian intervals are increasingly proposed for small studies, including confirmatory ones. We asked whether Bayesian intervals inherit this direction, and whether modelling the skewness removes it. Methods: Two registered simulation studies. Study 1 computed the independence-Jeffreys (fiducial) and Jeffreys-rule normal-likelihood intervals on exactly the 655 cells 10,000 datasets of two earlier studies, regenerated from archived seeds, and tested six hypotheses on direction, inheritance, first-order equivalence, opposed skew, null behaviour and validity. Study 2 compares posterior intervals from lognormal, gamma, Weibull and skew-normal likelihoods (correctly and incorrectly specified), stacking, and informative shape priors with Welch, bootstrap-t and a beta-bootstrap-t in 156 settings of positive-valued populations. Results: Study 1: the recomputed Welch and Hsu results matched the archived results exactly in all 655 cells. Both Bayesian intervals failed on the side of sign in 97.1–99.3% of determinate cells and on the same side as Welch in every cell where both were determinate (456 of 456 and 457 of 457); for groups of at least 30 the slope of their imbalance on Welch’s was 0.967–1.003. The prior changed the size slightly, never the side. Study 2: with a correctly specified skewed likelihood and groups of at least 20, the imbalance was at most half of Welch’s with coverage within 951.5 percentage points in 29 of 34 cells (median ratio to Welch 0.08), at a median width 1.33 times Welch’s. Misspecified fits were directionally worse than Welch in 4 of 120 (gamma), 7 of 132 (Weibull), 23 of 120 (lognormal) and 47 of 144 (skew-normal) cells; for the lognormal and skew-normal fits this happened most often at the largest group sizes. Stacking was not worse than the normal model in 58 of 66 cells, below the registered 90%. The effect of a wrongly centred shape prior decreased from 10 to 50 per group in 20 of 24 combinations. The correct model and stacking had smaller imbalances than bootstrap-t and beta-bootstrap-t in most cells. Conclusions: Noninformative priors do not change the side on which normal-likelihood intervals fail. In the families studied, a skewed likelihood removed the directional failure when it was the correct family, but a wrong skewed model could fail more than Welch’s interval, and more so as the groups grew. Credible intervals proposed for one-sided decisions in small studies should be evaluated for their one-sided error under plausible alternative families, not only under the model that produced them.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-18
DOI
https://doi.org/10.20944/preprints202611.1790.v1
Primary Topic
Statistical Methods and Bayesian Inference
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Do Bayesian Intervals Fail on the Same Side? Noninformative Priors, Skewed Likelihoods and the Direction of Small-Sample Interval Failure for a Difference in Means

Johan G.L. Verheyden
Preprints.org
Statistical Methods and Bayesian Inference
preprint

Do Bayesian Intervals Fail on the Same Side? Noninformative Priors, Skewed Likelihoods and the Direction of Small-Sample Interval Failure for a Difference in Means

Johan G.L. Verheyden
preprint en

Abstract

Background: When small-sample intervals for a difference of two arithmetic means fail under skewness, they fail mostly on one side, given by the sign of the skewness balance . Bayesian intervals are increasingly proposed for small studies, including confirmatory ones. We asked whether Bayesian intervals inherit this direction, and whether modelling the skewness removes it. Methods: Two registered simulation studies. Study 1 computed the independence-Jeffreys (fiducial) and Jeffreys-rule normal-likelihood intervals on exactly the 655 cells 10,000 datasets of two earlier studies, regenerated from archived seeds, and tested six hypotheses on direction, inheritance, first-order equivalence, opposed skew, null behaviour and validity. Study 2 compares posterior intervals from lognormal, gamma, Weibull and skew-normal likelihoods (correctly and incorrectly specified), stacking, and informative shape priors with Welch, bootstrap-t and a beta-bootstrap-t in 156 settings of positive-valued populations. Results: Study 1: the recomputed Welch and Hsu results matched the archived results exactly in all 655 cells. Both Bayesian intervals failed on the side of sign in 97.1–99.3% of determinate cells and on the same side as Welch in every cell where both were determinate (456 of 456 and 457 of 457); for groups of at least 30 the slope of their imbalance on Welch’s was 0.967–1.003. The prior changed the size slightly, never the side. Study 2: with a correctly specified skewed likelihood and groups of at least 20, the imbalance was at most half of Welch’s with coverage within 951.5 percentage points in 29 of 34 cells (median ratio to Welch 0.08), at a median width 1.33 times Welch’s. Misspecified fits were directionally worse than Welch in 4 of 120 (gamma), 7 of 132 (Weibull), 23 of 120 (lognormal) and 47 of 144 (skew-normal) cells; for the lognormal and skew-normal fits this happened most often at the largest group sizes. Stacking was not worse than the normal model in 58 of 66 cells, below the registered 90%. The effect of a wrongly centred shape prior decreased from 10 to 50 per group in 20 of 24 combinations. The correct model and stacking had smaller imbalances than bootstrap-t and beta-bootstrap-t in most cells. Conclusions: Noninformative priors do not change the side on which normal-likelihood intervals fail. In the families studied, a skewed likelihood removed the directional failure when it was the correct family, but a wrong skewed model could fail more than Welch’s interval, and more so as the groups grew. Credible intervals proposed for one-sided decisions in small studies should be evaluated for their one-sided error under plausible alternative families, not only under the model that produced them.

Preprints.org
Statistical Methods and Bayesian Inference
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon