Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon

Despite substantial research efforts evaluating how well large language models (LLMs) handle global cultural diversity, the mechanisms behind their cultural knowledge acquisition, particularly in multilingual settings, remain unclear.We study this question by investigating how cultural knowledge transfers across languages during the language adaptation of LLMs, a process where an LLM is continually pre-trained to learn another language.We introduce an interpretable framework to study this transfer, ensuring training data transparency and controlling transfer effects.Through a study of four non-Anglophonic cultures, we observe bidirectional cultural transfer between English and other high-resource languages, while low-resource languages primarily transfer knowledge to English with limited reverse flow.To explain this asymmetric phenomenon, we propose a frequency-based hypothesis: cultural knowledge appearing more frequently in the pretraining data transfers more easily, which is supported by empirical analysis of the training corpora.We hope our findings could inform future research on knowledge transfer and promote the development of culturally aware models, particularly for low-resource languages.