The design and evaluation of a visual narrated storytelling concept to improve end users understanding of explanations from a conceptual AI assistant

Explainable Artificial Intelligence methods and their presentation tend to dive deep into the technical details of complex models, making them difficult for domain users to understand. A particular challenge lies in enabling new users to build confidence and understanding of a digital assistant. We developed a visual narrated storytelling explainer concept and prototype to support a human operator's understanding of a conceptual AI-based digital assistant's decision-making process in managing an emergency in a future Urban Air Mobility traffic management scenario. The design exploration focused on how visual narrated storytelling can be applied to effectively convey the decision-making processes of a digital assistant supporting the proposed novel human role Urban Air Mobility Coordinator. The concept was designed using several dimensions: level of detail, temporal considerations, and modalities (visuals and narration). A mixed-method within-participant simulator experiment included qualitative and quantitative analysis and was conducted with eight Air Traffic Controllers. Participants' overarching experience, trust, and perceived appropriate level of detail of the storytelling concept was assessed. The storytelling concept helped users better understand how the digital assistant made decisions.