Optimization-Based Statistical Inference: Constrained Inverse Problems, Worst-Case Priors, and Kernel Regression

Optimization provides a worst-case framework for quantifying uncertainty in statistical inference, delivering robust and transparent performance guarantees. While this approach provides rigorous bounds, it cannot easily incorporate large-scale data or produce estimates at a prescribed confidence level. To bridge this gap, this thesis develops optimization-based methods that assimilate data while retaining worst-case robustness, exploring three different contexts: Ill-posed inverse problems, Bayesian inference with unknown priors, and Gaussian process regression. In the first, we introduce a new framework for frequentist, optimization-based intervals that provably achieves desired coverage. The framework unifies many previously proposed optimization-based intervals and disproves a conjecture dating back to 1965. In the second, we introduce data-likelihood constraints in Wald’s two-player zero-sum game, which renders the game computationally tractable and provides explicit certificates of minimax optimality. In the third, we develop new Gaussian process (GP) based methods for learning and solving partial differential equations and operator learning. In each setting, our GP algorithms achieve stronger convergence guarantees than existing machine-learning techniques without sacrificing predictive accuracy. Across these three settings, estimates for the unknown quantity (a finite-dimensional parameter, a prior distribution, or a function, respectively) are obtained as the solution to an optimization problem that characterizes either worst-case or minimax optimality, therefore contributing towards a single optimization-centric view of uncertainty quantification.