Public attitudes on performance for algorithmic and human decision-makers

This study explores public preferences for algorithmic and human decision-makers (DMs) in high-stakes contexts, how these preferences are shaped by performance metrics, and whether public evaluations of performance differ depending on the type of DM. Leveraging a conjoint experimental design, approximately <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>9,000</mml:mn></mml:math> respondents chose between pairs of DM profiles in two high-stakes scenarios: pretrial release decisions and bank loan approvals. The profiles varied by type (human vs. algorithm) and three metrics-defendant crime rate/loan default rate, false positive rate (FPR) among white defendants/applicants, and FPR among minority defendants/applicants-as well as an implicit fairness metric defined by the absolute difference between the two FPRs. The results show that efficiency was the most important performance metric in the respondents' evaluation of DMs, while fairness was the least prioritized. This finding is robust across both scenarios, key subgroups of respondents (e.g. by race and political party), and across the DM type under evaluation. Additionally, even when controlling for performance, we find an average preference for human DMs over algorithmic ones, though this preference varied significantly across respondents. Overall, these findings show that while respondents differ in their preferences over DM type, they are generally consistent in the performance metrics they desire.