Semismooth newton methods for risk-averse Markov decision processes

Inspired by semismooth Newton methods, we propose a general framework for designing solution methods with convergence guarantees for risk-averse Markov decision processes. Our approach accommodates a wide variety of risk measures by leveraging the assumption of Markovian coherent risk measures. To demonstrate the versatility and effectiveness of this framework, we design three distinct solution methods with proven convergence guarantees and that are not bounded to a specific risk measure. Furthermore, we establish that risk-averse policy iteration can be interpreted as an instance of semismooth Newton’s method. This result explains its superior convergence properties compared to risk-averse value iteration. Numerical experiments demonstrate the effectiveness of our algorithmic framework. The core contribution of our work lies in developing an algorithmic framework inspired by semismooth Newton methods to solve risk-averse Markov decision problems.