Explainable AI through the lens of Legitimate AI

Explainable AI (XAI) is frequently linked to responsible AI, under the assumption that it allows organisations to implement AI responsibly. However, with AI's growing use in civic settings, the number of people subjected to (partially) automated decisions rises, which should make us question if explanations from a responsible AI perspective would also benefit these subjects. We propose legitimate AI as an alternative and more relevant perspective to guide explainable AI (XAI) research focused on these subjects. If subjects are aided in their judgement of whether an AI is legitimate, they can serve as a societal check if the development and usage of an AI were done responsibly. In addition, explanations supporting that judgment enable them to decide to contest or accept a decision. We outline four aspects of legitimacy and their relevance to XAI, discussing a subject's potential questions for each aspect, highlighting topics that are currently under-explored. This perspective also uncovers the complexity of explanations aimed at empowering subjects, as such explanations need to reveal complex and intertwined topics through limited to no direct interaction.