Misplaced Capabilities: Evaluating the Risks of Anthropomorphism in Human-AI Interactions

The present research examines anthropomorphism, or human-like features, in conversational AI systems as a design element that facilitates human-AI interactions. The paper outlines how these human-like features intersect with user perceptions in ways that can co-create misplaced trust, and it explores ways to de-anthropomorphize AI systems. Using role-based prompts to elicit different anthropomorphic features within chatbot language and design, the study identifies and categorizes different types of anthropomorphism exhibited by large language models (LLMs), a necessary step towards evaluating the appropriate use of such features in technical systems. The role-based prompting process also provides a way to explore the stability of LLM responses. Ultimately, the paper explores how this approach could be incorporated into user studies to understand users' motivations, and it discusses the need for design interventions that can mitigate harms and biases hidden in human-AI interactions.