Advances in Bayesian Computation: Bridging Modern Machine Learning and Traditional Monte Carlo Methods

Bayesian inference requires solving integrals over probability spaces, but except for certain scenarios, they can only be approximated using Monte Carlo integration. Bayesian computation emerged specifically to develop efficient approximation methods, with sampling algorithms at its forefront. Nowadays, automatic differentiation and fast array computation software make numerical optimization the cutting edge in machine learning. These methods were introduced in Bayesian computation as variational inference and have been essential in accelerating Bayesian inference. This thesis is a methodological contribution to advancements in Bayesian computation, combining fast density approximation techniques with traditional asymptotically exact Monte Carlo methods. We start with a comprehensive review of Monte Carlo and variational inference techniques. Introduce an efficient, dimension-independent, and gradient-free sampling algorithm leveraging parallel computing architectures. Develop a novel Bayesian computation method that integrates flow matching with Markovian sampling, enhancing the exploration of complex target distributions through adaptive tempering mechanisms. Our work extends Bayesian nonparametric approaches to linear regression models, effectively handling outliers and heteroskedasticity via Dirichlet process mixtures. Finally, we present BlackJAX, a library for Bayesian inference, enabling researchers and practitioners to build and experiment with new algorithms seamlessly. These contributions collectively advance Bayesian computation, offering robust and efficient tools for empirical applications.