Ergodic optimal control problem of a success-runs Markov chain: a dissipativity approach

This paper studies the ergodic optimal control problem for a success-runs Markov chain. Our contributions are fourfold. First, we characterize the set of steady state-control pairs and identify a meaningful subclass of uniform (normal) steady states. Second, we analyse dissipativity of the problem by deriving a closed-form expression for the storage function, thereby linking the success-runs Markov chain to the stability theory of the associated deterministic difference equation that governs the state distribution. Third, we explicitly solve the long-run average reward problem, showing that the optimal policy is a stationary normal steady state and is independent of the initial condition. Finally, we demonstrate the applicability of the framework through two examples: a connection to a class of mean-field optimal control problems, and a solution to the optimal management of even-aged forest systems, which generalizes the classic Mitra–Wan forestry model.