Sequential transport maps using SoS density estimation and 𝛼-divergences

Transport-based density estimation methods are receiving growing interest because of their ability to efficiently generate samples from the approximated density. We further investigate the sequential transport maps framework proposed by Cui, Dolgov, and Zahm [J. Comput. Phys. 485 (2023), 31 pp.; Self-reinforced polynomial approximation methods for concentrated probability densities , March 2023], which builds on a sequence of composed Knothe–Rosenblatt (KR) maps. Each of those maps is built by first estimating an intermediate density of moderate complexity, and then by computing the exact KR map from a reference density to the precomputed approximate density. In our work, we explore the use of Sum-of-Squares (SoS) densities and α \alpha -divergences for approximating the intermediate densities. Combining SoS densities with α \alpha -divergence interestingly yields convex optimization problems which can be efficiently solved using semidefinite programming. The main advantage of α \alpha -divergences is to enable working with unnormalized densities, which provides benefits both numerically and theoretically. In particular, we provide a new convergence analysis of the sequential transport maps based on information geometric properties of α \alpha -divergences. The choice of intermediate densities is also crucial for the efficiency of the method. While tempered (or annealed) densities are the state-of-the-art, we introduce diffusion-based intermediate densities which permits to approximate densities known from samples only. Such intermediate densities are well established in machine learning for generative modeling. Finally we propose low-dimensional maps (or lazy maps) for dealing with high-dimensional problems and numerically demonstrate our methods on Bayesian inference problems and density estimation from data tasks.