Same Test, Different p: A Cross-Implementation Reproducibility Profile of the Paired t, Wilcoxon Signed-Rank, and Wilcoxon–Mann–Whitney Tests in R, SciPy, and Julia

We audit 5,461 deterministic fixtures across nine release-era stacks of R, SciPy, and Julia, using exact rational rank-test references and a validated 200-bit Student-t reference. Rates describe this constructed matrix, not real-world prevalence. On SciPy’s tie-free, zero-free exact signed-rank path, 35 distinct (fixture, alternative) pairs from 17 fixtures are not correctly rounded: 105 API calls across three equivalent zero-method settings. Four pairs (12 calls) return zero for positive probabilities near 2–8×10−17. The largest absolute error on the 35-pair set is 2.52 × 10−16; none changes a decision at α ∈ {0.05, 0.01, 0.001}. Maintainers diagnosed incorrect survival-function tail selection and merged a fix with a regression test the same day. Separately, R 4.6.1’s tied-exact rank-sum profile is correctly rounded on 1,134/1,260 calls, with maximum absolute error 2.35 × 10−15. Its exact and asymptotic signed-rank paths differ in zero handling. For paired t, the SciPy 1.17 Boost.Math migration changes correct-rounding counts from 112/270 to 125/270 against a fixed binary64 reference statistic followed by high-precision tail evaluation. Default decisions largely agree, while warning-free values outside [0, 1] require value-level checks. A second digest-pinned Docker environment reproduces all decision-level results while exposing ulp-level Julia sensitivity. The artifact distinguishes preserved aggregates, regenerated per-call data, and unrecovered runtime telemetry. Keywords: reproducibility, correct rounding, exact tests, Wilcoxon signed-rank test, Wilcoxon–Mann–Whitney test, paired t test, R, Python, Julia