MDPSolver: An Efficient Solver for Markov Decision Processes

MDPSolver is a Python package for easy and fast optimization of Markov Decision Process (MDP) problems.Our package is relevant to any decision-making problem where sequences of actions need to be taken and the outcome of each action is uncertain and revealed to the decision-maker one at a time.This specific type of decision-making problem is often denoted as an MDP or a discrete stochastic dynamic programming problem.For example, consider the sale and replenishment of goods from a wholesale inventory.Once a day, the wholesaler observes the amount of goods in the inventory and decides whether to create a replenishment order.Creating the order too early leads to an excess amount of goods, and unnecessary holding costs, whereas creating the order too late leads to shortage and lost sales.With MDPSolver, the wholesaler will be able to maximize profit by deriving optimized actions for each level of the remaining goods in the inventory.