Do Bayesian Intervals Fail on the Same Side? Noninformative Priors, Skewed Likelihoods and the Direction of Small-Sample Interval Failure for a Difference in Means
Background: When small-sample intervals for a difference of two arithmetic means fail under skewness, they fail mostly on one side, given by the sign of the skewness balance . Bayesian intervals are increasingly proposed for small studies, including confirmatory ones. We asked whether Bayesian intervals inherit this direction, and whether modelling the skewness removes it. Methods: Two registered simulation studies. Study 1 computed the independence-Jeffreys (fiducial) and Jeffreys-rule normal-likelihood intervals on exactly the 655 cells 10,000 datasets of two earlier studies, regenerated from archived seeds, and tested six hypotheses on direction, inheritance, first-order equivalence, opposed skew, null behaviour and validity. Study 2 compares posterior intervals from lognormal, gamma, Weibull and skew-normal likelihoods (correctly and incorrectly specified), stacking, and informative shape priors with Welch, bootstrap-t and a beta-bootstrap-t in 156 settings of positive-valued populations. Results: Study 1: the recomputed Welch and Hsu results matched the archived results exactly in all 655 cells. Both Bayesian intervals failed on the side of sign in 97.1–99.3% of determinate cells and on the same side as Welch in every cell where both were determinate (456 of 456 and 457 of 457); for groups of at least 30 the slope of their imbalance on Welch’s was 0.967–1.003. The prior changed the size slightly, never the side. Study 2: with a correctly specified skewed likelihood and groups of at least 20, the imbalance was at most half of Welch’s with coverage within 951.5 percentage points in 29 of 34 cells (median ratio to Welch 0.08), at a median width 1.33 times Welch’s. Misspecified fits were directionally worse than Welch in 4 of 120 (gamma), 7 of 132 (Weibull), 23 of 120 (lognormal) and 47 of 144 (skew-normal) cells; for the lognormal and skew-normal fits this happened most often at the largest group sizes. Stacking was not worse than the normal model in 58 of 66 cells, below the registered 90%. The effect of a wrongly centred shape prior decreased from 10 to 50 per group in 20 of 24 combinations. The correct model and stacking had smaller imbalances than bootstrap-t and beta-bootstrap-t in most cells. Conclusions: Noninformative priors do not change the side on which normal-likelihood intervals fail. In the families studied, a skewed likelihood removed the directional failure when it was the correct family, but a wrong skewed model could fail more than Welch’s interval, and more so as the groups grew. Credible intervals proposed for one-sided decisions in small studies should be evaluated for their one-sided error under plausible alternative families, not only under the model that produced them.
Authors
- Johan G.L. Verheyden (ORCID: https://orcid.org/0000-0002-6129-2794)
Publication Details
- Journal
- Preprints.org
- Published
- 2026-11-18
- DOI
- https://doi.org/10.20944/preprints202611.1790.v1
- Primary Topic
- Statistical Methods and Bayesian Inference
- Type
- preprint