Put Yourself in My Shoes: Revisiting the Moral Value of Algorithm Aversion Through Reciprocity and Vulnerability

Abstract This paper begins by exploring the phenomenon known as algorithm aversion , where users distrust AI and prefer human advice or decision-making even when they are aware of the algorithm’s superior performance. Current literature generally frames it as a misguided bias that harms decision accuracy and speed, likening it to a form of neo-Luddism. This view, however, overlooks the fact that the two groups (supporters and sceptics of algorithmic decisions) are speaking different moral languages: the supporters are outcome-orientated, arguing for accuracy and performance, while the sceptics offer a Kantian-position, that asks us to challenge the very precept that AI systems can be a decision-maker, or obligation-bearer , for decisions constrained by rights. To consider their position on the human-AI moral relationship, I take advantage of Korsgaard’s work on interspecies moral relationships, concluding that to be an obligation-bearer toward human right-holders, there needs to be a reciprocal reasons-giving relationship which AI in principle cannot fulfil. To meet objections that argue AI could eventually replicate the necessary moral agency requirements, I show that reciprocal relationships also call for constitutive symmetry , highlighting the importance of not only matching rationality, but also the vulnerability inherent in the human condition. With this account, algorithm sceptics are not misguided and have something morally important to say. This does not suggest eliminating AI entirely from decision processes. AI- assisted decision-making can be defended as long as a robust human-centred approach where genuine human control is upheld.