Ethical Markov Decision Processes

With the rise of automated agents in everyday life, integrating ethics in the decision-making processes of these agents becomes an important issue. In order to have a generic model which provides an expressive framework for specifying ethical behaviours, we propose in this article ethical Markov decision processes (E-MDPs), which extend classical MDPs with the explicit representation of moral values – positive or negative – that the agent’s decisions may promote or demote. We propose an algorithm for solving E-MDPs and illustrate our model on three distinct ethical frameworks: the Divine Command Theory framework, the Prima Facie Duties framework, and the Virtue Ethics framework, which respectively focus on prohibited states, mandatory state-action pairs, and exemplary trajectories represented by sets of transitions. As agents may not be alone in their environment and as decisions may be joint decisions instead, we also propose a multi-agent extension of E-MDPs based on stochastic games (E-SGs), where individual agents consider their impact in terms of moral values on the others. Finally, we implement the mono-agent model on an autonomous vehicle example previously proposed in the literature, aiming at evaluating how our generic model captures the different ethical frameworks. Then we implement the multi-agent model on a sequential allocation problem.