On Singular Bayesian Inference of Underdetermined Quantities—Part I: Invariant Discrete Ill-Posed Inverse Problems in Small and Large Dimensions

When the quantities of interest remain underdetermined a posteriori, we would like to draw inferences for at least one particular solution. Can we do so in a Bayesian way? What is a probability distribution over an underdetermined quantity? How do we get a posterior for one particular solution from a posterior for infinitely many underdetermined solutions? Guided by discrete invariant underdetermined ill-posed inverse problems, we find that a probability distribution over an underdetermined quantity is non-absolutely continuous, partially improper with respect to the initial reference measure but proper with respect to its restriction to its support. Thus, it is necessary and sufficient to choose the prior restricted reference measure to assign partially improper priors using e.g., the principle of maximum entropy and the posterior restricted reference measure to obtain the proper posterior for one particular solution. We can then work with underdetermined models like Hoeffding–Sobol expansions seamlessly, especially to effectively counter the curse of dimensionality within discrete nonparametric inverse problems. We show Singular Bayesian Inference (SBI) at work in an advanced Bayesian optimization application: dynamic pricing. Such a nice generalization of Bayesian–maxentropic inference could motivate many theoretical and practical developments.