The super moral status of artificial superintelligence

Abstract The potential emergence of artificial superintelligence has raised numerous ethical debates such as existential risks or value alignment. In this article, we address an underexplored discussion: the possibility of artificial superintelligence having a moral status superior to our own, what we have called super moral status. First, we make some preliminary remarks about the concepts of moral status and moral consideration. We present a broad definition of moral status and distinguish three other types of moral consideration. Second, we outline the developments that have taken place in two literatures, namely, the debate on postpersons in the ethics of human enhancement and the debate on the moral status of artificial intelligence. Third, we analyze the super moral status of artificial superintelligence from two dimensions of moral status theories: the foundationalist dimension and the gradualist dimension. We argue that the answer to the question of the super moral status of superintelligence does not depend on the type of foundationalist theory (whether it is ontological or relational), but on the type of gradualist theory (whether it is threshold-single, threshold-multiple, or scalar). Finally, we introduce the debunking argument: despite disagreement about whether artificial superintelligence would have super moral status, we do claim that the superior moral consideration of these entities would produce harms to humans. We have categorized three kinds of harms, these are, degradation in status, social (in)action and alienation, and domination.