Solidarity of Gibbs samplers: The spectral gap

Gibbs samplers are preeminent Markov chain Monte Carlo algorithms used in computational physics and statistical computing. Yet, their most fundamental properties, such as relations between convergence characteristics of their various versions, are not well understood. In this paper we prove the solidarity principle of the spectral gap for the Gibbs sampler: if any of the random scan or d! deterministic scans has a spectral gap then all of them have. Our methods rely on geometric interpretation of the Gibbs samplers as alternating projection algorithms and analysis of the rate of convergence in the von Neumann–Halperin method of cyclic alternating projections. As a byproduct of our analysis, we also establish that deterministic scan Gibbs, despite being nonreversible, share many robustness properties with reversible chains, including exponential inequalities and central limit theorems under the same conditions. In addition, we provide a quantitative result: if the spectral gap of the random scan Gibbs sampler decays with dimension at a polynomial rate β, then the rate is no worse than 2β+2 for any deterministic scan.