Examining the Feasibility of Large Language Models as Survey Respondents

This study examines the potential of large language models (LLMs) to substitute for human respondents in survey research. Surveys serve as essential tools in fields like social science, marketing, and policy-making; however, traditional methods often require considerable time and costs. LLMs present a promising alternative to mitigate these burdens, though their reliability—particularly outside of U.S. contexts—remains uncertain. This study focuses on surveys conducted in Japan, comparing the responses generated by LLMs to those of actual Japanese participants. Our analysis reveals notable discrepancies due to inherent biases in LLMs, though adjusting the models to better align with specific personas can partially enhance the accuracy of simulated responses. We emphasize the need for further research to fully understand the capabilities and limitations of LLMs, aiming to refine their application in diverse areas such as social sciences, marketing, and policy decision-making.