Moral Agents Unlike Us (Extended Abstract)

Suppose AI developers succeed in creating advanced non-conscious artificial moral agents—AI systems that meet all the standard criteria for moral agency yet lack phenomenal consciousness. Initially, it might seem that we should be indifferent between human moral agents and artificial moral agents in moral decision-making contexts. In this paper, I argue that we have grounds for requiring certain decisions to be made by human moral agents. I note two asymmetries that arise between human moral agents and artificial moral agents in virtue of artificial moral agents’ lack of phenomenal consciousness: a moral status asymmetry and an affective asymmetry. I then argue that these asymmetries lead to two factors that have bearing on when we should not be indifferent between human moral agents and artificial moral agents: relationships and responsibility. I conclude by analyzing concrete cases and considering implications for existing AI systems.