Stochastic control with controlled information

This thesis addresses the problem of stochastic control, where the control of available information is permitted. We examine three distinct stochastic control problems, each with a unique aspect of controlled information. These problems share a common Bayesian approach to estimation, wherein the prior is known and subsequently updated to account for the impact of control on the estimation of the unobservable component(s). The first part of the thesis addresses a problem of optimal control with separable drift uncertainty. We consider a controlled state process, that is influenced by a hidden, static random variable which allows for parameter uncertainty of the drift coefficient function. The initial problem is non-Markovian, but employing techniques of Bayesian inference, we perform a Markovian lift. We prove that the strong and weak formulations of the problem agree in value and construct 𝜀-optimal controls using the stochastic Perron method. A numerical example is provided to illustrate the learning effect of the optimal control. The second part of the thesis concerns an infinite horizon linear quadratic control problem with additional, costly information acquisition. Additional to a baseline of information, the controller can continuously acquire information about a hidden Ornstein–Uhlenbeck process. We employ once again the techniques of Bayesian inference to derive a Markovian lift of the problem, but here specifically the Kalman–Bucy filter is used to estimate the hidden Ornstein–Uhlenbeck process. The problem is solved via a combination of an ansatz for the value function and a proof of regularity of a reduced value function. The qualitative properties of the optimally controlled state process, the optimal control and the optimal information acquisition rate are discussed, with a focus of their properties at the long-time equilibrium. The third part of the thesis deals with the Merton problem of optimal investment under power utility. We consider a one-dimensional risky asset, whose drift is unobservable and Markov-modulated. At any point in time, the investor can acquire expert opinions about the drift of the asset, but at a cost, consisting of a fixed fee and a fee proportional to the quality of the expert opinion. Again, we take a Bayesian view and use the Wonham filter to perform a Markovian lift of the problem. Mathematically, the problem falls into the category of mixed (classical and impulse) control. We prove that the value function is the unique viscosity solution of the HJB equation and construct an optimal control with an explicit construction of the impulse control.