Adaptive tuning of Hamiltonian Monte Carlo methods

• ATune performs adaptive parameter tuning in (Generalized) Hamiltonian Monte Carlo • ATune detects a system specific integrator with reliable randomized parameters • It is computationally efficient and easy to implement in Bayesian analysis software • ATune enhances sampling performance in real-world applications • Extensive tests show its robustness across diverse Bayesian model classes With the recently increased interest in probabilistic models, the efficiency of an underlying sampler becomes a crucial consideration. Hamiltonian Monte Carlo is one popular option for models of this kind. Performance of the method, however, strongly relies on a choice of parameters associated with an integration approach for Hamiltonian equations. Up to date, such a choice remains mainly heuristic or introduces time complexity. We propose a novel computationally inexpensive and flexible approach (we call it Adaptive Tuning or ATune) that, by combining a theoretical analysis of the multivariate Gaussian model with simulation data generated during a burn-in stage of a Hamiltonian Monte Carlo simulation, detects a system specific splitting integrator with a set of reliable sampler’s hyperparameters, including their credible randomization intervals, to be readily used in a production simulation. The method automatically eliminates those values of simulation parameters which could cause undesired extreme scenarios, such as resonance artifacts, low accuracy or poor sampling. The new approach is implemented in the in-house software package HaiCS , with no computational overheads introduced in a production simulation, and can be easily incorporated in any package for Bayesian inference with Hamiltonian Monte Carlo. The tests on popular statistical models reveal the superiority of adaptively tuned standard and generalized Hamiltonian Monte Carlo methods in terms of stability, performance and accuracy over conventional Hamiltonian Monte Carlo tuned heuristically and coupled with the well-established integrators. We also claim that the generalized Hamiltonian Monte Carlo is preferable for achieving high sampling performance. The efficiency of the new methodology is assessed in comparison with state-of-the-art samplers, e.g. the No-U-Turn-Sampler, in real-world applications, such as endocrine therapy resistance in cancer, modeling of cell-cell adhesion dynamics and influenza A epidemic outbreak.