Partition function approach to non-Gaussian likelihoods: macrocanonical partitions and replicating Markov-chains

Monte-Carlo techniques are standard numerical tools for exploring non-Gaussian and multivariate likelihoods. Many variants of the original Metropolis-Hastings algorithm have been proposed to increase the sampling efficiency. Motivated by Ensemble Monte Carlo we allow the number of Markov chains to vary by exchanging particles with a reservoir, controlled by a parameter analogous to a chemical potential <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mi>μ</mml:mi> </mml:math> , which effectively establishes a random process that samples microstates from a macrocanonical instead of a canonical ensemble. In this paper, we develop the theory of macrocanonical sampling for statistical inference on the basis of Bayesian macrocanonical partition functions, thereby bringing to light the relations between information-theoretical quantities and thermodynamic properties. Furthermore, we propose an algorithm for macrocanonical sampling, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mtext mathvariant="monospace">𝙰𝚟𝚊𝚕𝚊𝚗𝚌𝚑𝚎 𝚂𝚊𝚖𝚙𝚕𝚒𝚗𝚐</mml:mtext> </mml:math> , and apply it to various toy problems as well as the likelihood on the cosmological parameters <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:msub> <mml:mi>Ω</mml:mi> <mml:mi>m</mml:mi> </mml:msub> </mml:math> and <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mi>w</mml:mi> </mml:math> on the basis of data from the supernova distance redshift relation.