Developing user-centered system design guidelines for explainable AI: a systematic literature review

Abstract The rapid advancement of AI technology has led to increasingly complex and opaque systems, creating a critical need for explainable AI (XAI) that enhances transparency and user trust. Despite extensive research on XAI methods and applications, the user-centered design (UCD) process for XAI remains fragmented and unclear. This systematic review analyzes 27 studies from 2020 to 2024 to develop a comprehensive framework for user-centered XAI system design (UCXAISD). We identify five key stages: contextual inquiry, explanation needs identification, XAI method selection, user interface design, and evaluation and refinement. Our framework aligns with traditional UCD processes while incorporating specialized elements for XAI, including user-centricity, transparency, and actionability. For each stage, we provide evidence-based guidelines derived from the literature. The framework serves as a blueprint for developing XAI systems that balance technical sophistication with user needs.