Effective but untrustworthy: How artificial intelligence bias opposing human bias affects judgments

Today, people make judgments with the help of artificial intelligence (AI) assistance in many situations, such as medical diagnoses. Although many studies have examined the effects of AI assistance, they have mainly focused on aspects of AI (e.g., AI's accuracy). Here, we emphasize the importance of interactions between AI and human biases. A highly accurate AI may not always be a promising intervention; rather, AI with biases (especially in the direction opposite to individuals' biases) may work effectively because AI's biases may cancel out individuals' biases (e.g., individuals' overestimation bias may be corrected by AI's underestimation bias). We investigated these is-sues using a simple perceptual task assuming medical judgments. First, computer simulations showed that appropriate AI assistance would differ depending on individuals' prior beliefs. Behavioral experiments demonstrated that AI with biases in the direction opposite to participants' biases could effectively reduce their biases. However, participants tended to evaluate AI with biases in the same direction as their own and considered it more trustworthy. Our theoretical and empirical results raise questions about conventional beliefs that more accurate, trustworthy AI should be better. Our findings will provide practical implications for designing AI as a collaborator of people.