On the Relation of Emotion and Moral Capacity in Artificial Intelligence Technologies

The unpreceded abilities of AI technologies have led to the emergence of new ethical issues; among them is the possibility of the moral agency of AI artifacts. There are many questions around this subject, including what are the necessary and sufficient conditions of being a moral agent? How can we examine those conditions in artifacts? What levels and degrees of agency are possible for artifacts? And what level of moral agency is proper for allocating a certain task to AI artifacts? There are wide discussions about factors that figure in the moral capacities of AI artifacts, and emotions are one of the frequently referred factors. Emotions are directly or indirectly relevant for examining AI’s moral status. In this paper, we focus on the relation between emotions and moral capacities in AI technologies. Our main question is whether emotions play a positive role in the improvement of moral capacities or a negative role. We extract and articulate four arguments in defense of the positive role of emotions in the enhancement of AI moral capacities, including arguments from moral sensitivity, bounded rationality, risk assessment, and culpability. Then, we present other four arguments in defense of the negative role of emotions in moral capacities including arguments from emotional hijacking, deceptive emotions, anthropomorphism and dehumanization paradox, and moral deskilling. Finally, we analyze the debate by clarifying the point of contest between the two mentioned camps and discuss serious challenges of designing emotional AI.