Variational Bayesian inference for models with nuisance parameters and an intractable likelihood

Abstract A primary challenge in Bayesian analysis lies in computing the posterior distribution of model parameters, especially for models with a large number of parameters or intractable likelihoods. Often, the focus is on a subset of parameters, with the remainder regarded as nuisance parameters introduced for computational convenience. This complexity necessitates refined computational methods. Variational Bayesian inference (VB) has emerged as a powerful solution, enhancing computational efficiency by recasting inference as an optimization problem within a family of tractable distributions. However, common VB techniques sometimes fall short, especially for models with nuisance parameters or intractable likelihoods. After identifying characteristics of suboptimal VB methods, we build upon the Hybrid Variational Bayes (HVB) approach introduced by Loaiza-Maya et al. (2022) and develop an extended and unified HVB framework designed to achieve more precise Bayesian inference in such scenarios. Through theoretical exploration and a series of illustrative examples, our approach demonstrates notable improvements over traditional VB methods.