Posterior Inference via Flexible Variational Approximations with Conditional Independence Structure

We consider the problem of finding structured variational approximations of Bayesian posterior distributions with known conditional independence structures. Recent literature on the subject considers parsimonious parametrizations of distributions which are either Gaussian or close to Gaussian with skew or otherwise non–normal marginals, via the precision matrix. In this work, we develop a structured variational approximation based on a generalization of the precision matrix involving the Hessian matrix of the variational density. We characterize the problem of encoding this constraint on the Hessian to build approximations based on sequences of compositions of functions of arbitrary length for posterior conditional independence structures corresponding to chordal graphs, and develop conditions under which graphically identical component functions may be repeated while preserving sparsity. This methodology is applied to posterior inference in Bayesian logistic regression with a shrinkage inducing prior and a Bayesian logistic random intercepts model. An empirical study is conducted on high-dimensional simulated and popular benchmark datasets, with comparisons to Gaussian variational approximations and Normalizing Flows. Supplemental materials are available online.