In the Eyes of the User: A Systematic Literature Review of User Perceptions in Human-XAI Interaction

The widespread integration of AI systems across diverse areas of society and the economy offers considerable potential to assist humans in decision-making processes. However, as these systems often exhibit a high degree of complexity and act like black boxes in their decision-making, the evolving research field of Explainable AI (XAI) attempts to make the internal functioning and decision-making mechanisms of AI systems comprehensible to humans. This raises the question of how such XAI methods are perceived by humans. To gain a comprehensive understanding of user perceptions in human-XAI interaction, we conduct a systematic literature review. Our findings reveal five categories of user perception, which are influenced by multiple factors and result in behavioral and performance outcomes. This study contributes to a holistic understanding of user perceptions in the context of Explainable AI, benefiting both theoretical and practical aspects of information systems and XAI research.