Navigating Uncertainty: A User-Perspective Survey of Trustworthiness of AI in Healthcare

This article offers an extensive survey of one of the fundamental aspects of the trustworthiness of AI in healthcare, namely uncertainty, focusing on the large panoply of recent studies addressing the connection between uncertainty, AI, and healthcare. The concept of uncertainty is a recurring theme across multiple disciplines, with varying focuses and approaches. Here, we focus on the diverse nature of uncertainty in medical applications, emphasizing the importance of quantifying uncertainty in model predictions and its advantages in specific clinical settings. Questions that emerge in this context range from the guidelines for AI integration in the healthcare domain to the ethical deliberations and their compatibility with cutting-edge AI research. Together with a description of the main specific works in this context, we also discuss that, as medicine evolves and introduces novel sources of uncertainty, there is a need for more versatile uncertainty quantification methods to be developed collaboratively by researchers and healthcare professionals. Finally, we acknowledge the limitations of current uncertainty quantification methods in addressing the different facets of uncertainty within the medical domain. In particular, we identify from this survey a relative paucity of approaches that focus on the user’s perception of uncertainty and accordingly of trustworthiness.