Social Bots Meet Large Language Model: Political Bias and Social Learning Inspired Mitigation Strategies

Recent advances in the large language models (LLM) have empowered traditional bots to gain human-level intelligence and exhibit human-like social behaviors, giving rise to a new form of LLM-driven social agents. However, the inherent limitations in LLMs could potentially result in politically biased behaviors of these agents, posing unexpected risks to human society. While great efforts have been made to examine political bias and related concerns in traditional bots and LLMs, little is known about the existence, unique characteristics, underlying origins, and potential mitigation strategies of this bias in LLM-driven social agents. To address this gap, we systematically assess political bias in LLM-driven social agents, by examining how it emerges as these agents self-reflect, communicate, and understand others during social interactions. Through designing and implementing social experiments, we discover that this bias consistently manifests in the social behaviors of agents driven by diverse LLMs, across nine key political topics. Inspired by the social learning theory, we propose to mitigate political bias by guiding these agents to emulate how humans learn to behave. By incorporating self-regulated and role-model learning processes, we reduce their political bias by 4.89% to 51.26% across diverse LLMs and topics, demonstrating the effectiveness and generalizability of the proposed strategy. This study not only advances the understanding of political bias in emerging LLM-driven agents, but also offers insights into harnessing social bots for social good.