Simulating conversations on social media with generative agent-based models

Large Language Models (LLMs) can generate realistic text resembling human-produced content. However, the ability of these models to simulate conversations on social media is still less explored. To investigate the potential and limitations of simulated text in this domain, we introduce network-simulator, a system to simulate conversations on social media. First, we simulate the macro structure of a conversation using Agent-Based Modeling (ABM). The generated structure defines who interacts with whom, the type of interaction, and the agent’s stance on the topic of the conversation. Subsequently, using the simulated interaction structure, our system generates prompts conditioned on the simulation variables, producing texts that are conditioned on the parameters of the predefined structure, guiding a micro simulation process. We compare human conversations with those simulated by our system. Based on stylistic and model-based metrics, we found that our system can simulate conversations on social media in various dimensions. However, we detected differences in metrics related to the predictability of text production. Furthermore, we examine the effect of true and false framings within simulated conversations, revealing that simulated discussions surrounding false information exhibit a more negative collective sentiment bias than those based on true content.