Explainable AI: definition and attributes of a good explanation for health AI
Abstract Proposals of artificial intelligence (AI) solutions based on more complex and accurate predictive models are becoming ubiquitous across many disciplines. As the complexity of these models increases, there is a tendency for transparency and users’ understanding to decrease. This means accurate prediction alone is insufficient to make an AI-based solution truly useful. For the development of healthcare systems, this raises new issues for accountability and safety. How and why an AI system made a recommendation may necessitate complex explanations of the inner workings and reasoning processes. While research on explainable AI (XAI) has grown significantly in recent years, and the demand for XAI in medicine is high, determining what constitutes a good explanation is ad hoc and providing adequate explanations remains a challenge. To realise the potential of AI, it is critical to shed light on two fundamental questions of explanation for safety–critical AI such as health-AI that remain unanswered: (1) What is an explanation in health-AI? And (2) What are the attributes of a good explanation in health-AI? In this study and possibly for the first time we studied published literature, and expert opinions from a diverse group of professionals reported from a two-round Delphi study. The research outputs include (1) a proposed definition of explanation in health-AI, and (2) a comprehensive set of attributes that characterize a good explanation in health-AI.
Authors
- Olga Kostopoulou (ORCID: https://orcid.org/0000-0001-9643-0838)
- Max Marsden (ORCID: https://orcid.org/0000-0002-7147-6861)
- Keri Grieman
- Ben Glocker (ORCID: https://orcid.org/0000-0002-4897-9356)
- Evangelia Kyrimi (ORCID: https://orcid.org/0000-0001-6727-2279)
- David S. Berman (ORCID: https://orcid.org/0000-0002-5382-1668)
- Burkhard Schäfer (ORCID: https://orcid.org/0000-0002-6025-4593)
- William Marsh (ORCID: https://orcid.org/0000-0003-0212-6363)
- Kudakwashe Dube
- David A. Lagnado (ORCID: https://orcid.org/0000-0002-6845-8830)
- Constantine Gatsonis
- Somayyeh Mossadegh
- Danielle Sent (ORCID: https://orcid.org/0000-0002-4703-5345)
- Paul Curzon (ORCID: https://orcid.org/0000-0002-1301-9734)
- Ari Ercole (ORCID: https://orcid.org/0000-0001-8350-8093)
- Tim Miller (ORCID: https://orcid.org/0000-0003-4908-6063)
- Scott McLachlan (ORCID: https://orcid.org/0000-0002-2528-8050)
- Nigel Tai (ORCID: https://orcid.org/0000-0001-8493-9063)
- Vince I. Madai (ORCID: https://orcid.org/0000-0002-8552-6954)
- Sankalp Tandle (ORCID: https://orcid.org/0009-0000-8210-728X)
- Zane Perkins (ORCID: https://orcid.org/0000-0003-4807-8803)
- Hamit Soyel (ORCID: https://orcid.org/0000-0002-0305-574X)
- Francesco Leofante (ORCID: https://orcid.org/0000-0001-8245-9429)
- Myura Nagendran (ORCID: https://orcid.org/0000-0002-4656-5096)
- Jon Williamson (ORCID: https://orcid.org/0000-0003-0514-4209)
- Alexander Gimson
- Jared M. Wohlgemut (ORCID: https://orcid.org/0000-0001-8276-0465)
- Rebecca S. Stoner (ORCID: https://orcid.org/0000-0001-6287-780X)
- Dimitrios Letsios (ORCID: https://orcid.org/0000-0002-3258-7585)
- David J. Glass (ORCID: https://orcid.org/0000-0001-6187-4164)
- Crina Groşan (ORCID: https://orcid.org/0000-0003-1049-2136)
- Stefan Buijsman (ORCID: https://orcid.org/0000-0002-0004-0681)
- Ali Shafti (ORCID: https://orcid.org/0000-0001-7459-820X)
- Amitava Banerjee (ORCID: https://orcid.org/0000-0001-9241-3555)
- Declan P. O’Regan (ORCID: https://orcid.org/0000-0002-0691-0270)
- Erhan Pisirir
- Owain Jones (ORCID: https://orcid.org/0000-0003-4548-8735)
- Shalmali Joshi (ORCID: https://orcid.org/0009-0002-6594-4902)
- Dylan Morrissey (ORCID: https://orcid.org/0000-0001-7045-3644)
Institutions
- Queen Mary University of London (GB)
- King's College London (GB)
- Barts Health NHS Trust (GB)
- Royal London Hospital (GB)
- University College London (GB)
Publication Details
- Journal
- AI and Ethics
- Published
- 2025-03-04
- DOI
- https://doi.org/10.1007/s43681-025-00668-x
- Citations
- 4
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- article
- Field-Weighted Citation Impact
- 9.92
Funders
- Royal Academy of Engineering