An Uninterpretable Right: Legal and Practical Limits of the Right to an Explanation

The European Union’s AI Act introduces a right of explanation for decisions made by high-risk AI systems. Similar rights appear in legal instruments such as the General Data Protection Regulation (GDPR), each invoking explanation as a means of promoting transparency and accountability in automated decision-making. This paper argues that such rights offer a reassuring but ultimately misleading response to a deep epistemological mismatch between legal and machine reasoning. Legal explanations presuppose causal intelligibility and normative justification – features that most modern machine learning systems cannot meaningfully provide. Drawing on recent developments such as the Dun & Bradstreet ruling of the Court of Justice of the European Union, which clarified the contours of explanation rights under the GDPR, we demonstrate how technical explanation methods such as counterfactuals and contrastive reasoning fail to bridge the gap between statistical inference and normative legitimacy. This paper proposes a normative four-part framework – the ‘legitimacy bridge’ – to assess the legal adequacy of explanations, arguing that transparency must be linked to justification. While explanations may assist in system auditing, they are insufficient as a legal foundation for algorithmic accountability.