Estimating a probability density from imperfect data

Abstract Training generative models in data-scarce scientific domains poses a significant challenge, particularly when direct samples from the target distribution are unavailable. In many settings, including geophysics, medical imaging, and fluid dynamics, only indirect observations can be collected, arising from non-invertible or ill-conditioned transformations of latent variables of interest. This paper addresses the problem of learning a latent distribution $\pi_{\bfx}(\bfx)$ given samples drawn from its push-forward $\pi_{\bfy}(\bfy)$, where $\bfy = F(\bfx)$ and $F$ is not invertible. Unlike methods that assume access to both the latent variables and their projections, we consider the more realistic and challenging setting where only the projections $\bfy_j$ are observed, and the latent variables $\bfx_j$ are never seen. To tackle this ill-posed inverse problem, we study two complementary approaches: a simulator-free maximum-entropy estimator and simulator-based calibration using sample-based discrepancies such as MMD and adversarial losses. The maximum-entropy approach is useful when the forward operator is only mildly ill-posed, while simulator-based calibration provides additional structural information for severely ill-posed problems with large null spaces. The resulting models estimate latent distributions that are consistent with observed marginals and, when available, domain constraints. We provide empirical evidence that these approaches can recover meaningful latent distributions across multiple domains, even when the forward operator $F$ is many-to-one and ground truth samples are unavailable.