Dialogue-Based XAI Approaches: Exploring the Continuum from Static Explanations to Free-Form Chat Interfaces

It is increasingly recognized that end users have individual and specific explanation needs in the context of eXplainable Artificial Intelligence (XAI). The XAI field is thus shifting towards personalized and co-constructive explanations for non-technical stakeholders. Therefore, conversational XAI systems are proposed, aiming to provide a natural conversation between the user and the XAI system and promise to improve user understanding through the adaptive nature of bidirectional explanations. However, evidence on the effectiveness of such systems, particularly those utilizing large language models (LLMs), remains limited. In this study, we explore dialogue interfaces using a three-phase experiment capturing the user's objective and subjective understanding in conditions ranging from static explanation reports to guiding chatbots. Our findings demonstrate the effectiveness of a single- and two-prompt LLM mechanism for intent mapping and suggest that participants who engaged more frequently, especially with feature-specific questions, achieved higher understanding scores. We also uncover that participants prefer to select suggested explanations rather than type their own questions and argue that successful chat interfaces must extend beyond simple question-to-XAI method mapping to enhance model understanding.