A behavioral economics approach to programming autonomous systems in moral dilemmas

Abstract Autonomous systems (ASs) will eventually make many decisions in moral dilemmas without human oversight. How humans will respond to the presence of ASs and whether humans will accept them will crucially depend on how ASs’ decisions are programmed. For example, how moral dilemmas are approached in the programming of autonomous vehicles (AVs) is stated as an important factor by potential adopters of this technology. After all, how risks are assigned between different road users will have a strong impact on the socioeconomic structures of society’s mobility. Until now, research on preferences for the programming of ASs in moral dilemmas confronts participants in many instances with life-and-death scenarios based on abstract philosophical thought experiments. This paper offers a first step in evaluating how new knowledge in this area can be acquired through the combination of disciplines. Three empirical studies illustrate the potential advantages of a behavioral economics approach for this string of research. We compare choices in high-stakes hypothetical vignette dilemmas with choices in structurally equivalent low-stakes incentivized experiments. We find that, especially for dilemmas with self-interest, incentivized experiments can provide important additional information, as people tend to overstate their altruistic tendencies in hypothetical dilemmas.