Dynamic programming principle and Hamilton–Jacobi–Bellman equation for optimal control problems with uncertainty

We study the properties of the value function associated with an optimal control problem with uncertainties, known as average or Riemann–Stieltjes problem. Uncertainties are assumed to belong to a compact metric probability space, and appear in the dynamics, in the terminal cost and in the initial condition, which yield an infinite-dimensional formulation. By stating the problem as an evolution equation in a Hilbert space, we show that the value function is the unique lower semicontinuous proximal solution of the Hamilton–Jacobi–Bellman (HJB) equation. Our approach relies on invariance properties and the dynamic programming principle.