An Optimization Framework For Data Enrichment: A First Principles Approach To Designing Optimal Smoothers and Splines

We develop a general framework for state estimation in systems modeled with stochastic dynamics and noisy measurements. Our approach is based on maximum likelihood estimation and employs a variety of techniques from optimization theory including the calculus of variations and discrete optimization to derive optimality conditions for many different types of problems. We make no rigid assumptions on the form of the mapping from measurements to state-estimate or on the distributions of the random processes, making the framework more general than existing techniques such as splines, Kalman smoothing or Gaussian Process Regressions, which are subject to limiting restrictions and modeling challenges. The framework is first applied to a discrete fluid dynamical system where the position of a particle moving in a turbulent flow is estimated. The optimal solution is naturally interpreted as a digital filter and its performance is compared to related methods in the literature to demonstrate efficacy. In a second application the framework is applied to general continuous time systems in which the optimal solution that arises is interpreted as a continuous time spline, the structure and temporal dependency of which adapts to the system dynamics and the distributions of the process and measurement noise automatically. We demonstrate the utility and generality of our splines via illustrative examples that render both linear and nonlinear data filters depending on the particular system. In a final application multi-parameter continuous systems are analyzed to generate state estimates on high dimensional system domains. The optimal solution arising in the final investigation is generated using a superposition of physically motivated basis functions, both derived and fit to measurements using our framework, and the solution is interpreted as an adaptive B-spline techniques. Following the initial general treatment, the multi-parameter solution is used to produce state estimates on real open-source population data measured in Ghana over latitude and longitude as an illustrative example.