Evaluating Cross-National Value Alignment in Large Language Models: Challenges and Limitations of Survey-Based Approaches

Large language models (LLMs) often exhibit cultural biases, raising questions about their alignment with human values across different countries. This study examines the cross-national value alignment of LLMs, focusing on responses from the United States, China, and Egypt using the World Values Survey (WVS) dataset. We evaluate the alignment of LLM responses to national value distributions and analyze linguistic features such as point of view, justification, and hedging tendencies. Our findings reveal that LLMs display a default alignment towards US-centric values, with varying degrees of success in reflecting the values of other nations. Furthermore, we highlight the methodological challenges and potential ecological fallacies in applying survey-based cultural research to LLM evaluation. This research underscores the need for nuanced approaches in assessing and enhancing the cultural alignment of LLMs.