Human-Aligned AI Must Counter Overtrust

The psychological reality of human baseline overtrust in AI has been increasingly recognized in recent years. Here, we argue that for human-aligned AI to successfully advance human goals and welfare, in many contexts it will need to gauge – and counter if need be – human propensities for overtrust. We briefly summarize our original program of research documenting overtrust in contexts of grave decisionmaking, and provide suggestions for ways that artificial agents might be prepared to estimate and respond to human overtrust.