Utilizing Large Language Models for Social Simulations: Responding to the Degree of Freedom Questionnaire with ChatGPT

The evolution of artificial intelligence has enabled the simulation of complex scenarios that are costly and challenging to conduct in real life, significantly impacting fields such as chemistry, epidemiology, and climate prediction, especially in drug discovery. We hypothesized that simulations could be applied not only to the physical and quantitative aspects of the world, as previously mentioned, but also to the qualitative domains of human society. To achieve this, we presented ChatGPT with questions from the Press Freedom Index survey and compared its responses to those of human experts. The results revealed a significant correlation between ChatGPT’s outputs and the actual survey results, though notable differences emerged based on the language used. This indicates the considerable potential of LLMs like ChatGPT as prototyping tools for social science experiments. However, to ensure unbiased performance and content across different cultures, meticulous training and evaluation in each language are necessary. Our study highlights the potential of large language models to serve as preliminary sandboxes for simulations across various fields, extending beyond technical applications to include social sciences.