Why We Need Epistemically, Not Morally, Trustworthy AI: Decision Support and the Need for Collective Accountability

Abstract Calls for “trustworthy AI” have become ubiquitous in policy and industry, yet the term remains conceptually underspecified. In earlier work (Dorsch & Deroy, 2024), we argued that moral trustworthiness is neither possible nor necessary for AI decision-support systems (AI-DSS), and indeed unethical to pursue, since it risks the category mistake of attributing benevolence to systems that lack it. This paper revisits that claim in light of recent critiques and developments, arguing that once moral trustworthiness is set aside, the central normative question shifts from whether AI systems can be trusted to how their epistemic influence on human decision-making ought to be justified and governed. We begin by situating our earlier thesis alongside complementary arguments that converge on the dangers of anthropomorphising AI. Second, we respond to critics who seek residual roles for moral trustworthiness and who clarify the epistemic conditions of trust. Third, we advance an account of epistemic trustworthiness as a property of AI outputs, rather than a property of the systems themselves. We contend that guaranteeing epistemic trustworthiness depends on human institutions, and is necessary to mitigate vulnerabilities introduced by integrating AI-DSS into hybrid decision-making. We connect this property to legal frameworks, showing how the EU AI Act’s ‘Right to Explanation’ codifies it as a necessary condition for deploying AI in high-risk domains.