Beta-Bernoulli Filtering and Linear Feedback Control Based Step-Size Adaptation for HMC and MALA

In this paper, we propose step-size adaptation methods for the Hamiltonian Monte Carlo (HMC) and Metropolis-adjusted Langevin algorithms (MALA). The adaptation procedures consist of an acceptance rate estimator which is implemented as a Bayesian filter on the observed acceptance indicator sequence. This sequence is modeled as a Bernoulli sequence with a time-varying probability, and its distribution is represented by a beta distribution. Therefore, the resulting filter is called the Beta-Bernoulli filter. The acceptance rate is then controlled to the desired target acceptance rate using a linear feedback controller. The resulting adaptation mechanism is experimentally evaluated in practical MCMC sampling tasks.