Trusting AI: does uncertainty visualization affect decision-making?

Introduction Decision-making based on AI can be challenging, especially when considering the uncertainty associated with AI predictions. Visualizing uncertainty in AI refers to techniques that use visual cues to represent the level of confidence or uncertainty in an AI model's outputs, such as predictions or decisions. This study aims to investigate the impact of visualizing uncertainty on decision-making and trust in AI. Methods We conducted a user study with 147 participants, utilizing static classic gaming scenarios as a proxy for human-AI collaboration in decision-making. The study measured changes in decisions, trust in AI, and decision-making confidence when uncertainty was visualized in a continuous format compared to a binary output of the AI model. Results Our findings indicate that visualizing uncertainty significantly enhances trust in AI for 58% of participants with negative attitudes toward AI. Additionally, 31% of these participants found uncertainty visualization to be useful. The size of the uncertainty visualization was identified as the method that had the most impact on participants' trust in AI and their confidence in their decisions. Furthermore, we observed a strong association between participants' gaming experience and changes in decision-making when uncertainty was visualized, as well as a strong link between trust in AI and individual attitudes toward AI. Discussion These results suggest that visualizing uncertainty can improve trust in AI, particularly among individuals with negative attitudes toward AI. The findings also have important implications for the design of human-AI decision-support systems, offering insights into how uncertainty can be visualized to enhance decision-making and user confidence.