A systematic review of bias detection methods for non-English word embeddings and language models

Abstract Biases in applications of machine learning and artificial intelligence are a major limitation of these applications. Stereotypes of the society are reflected in different types of applications, including image generation, machine translation or CV ranking. This is in particular also the case for language models and word embeddings, encoding human language as mathematical vectors. Research addressing the challenging problem of detection (and mitigation) of the bias in these embeddings is often conducted for the English language. However, the stereotypes encoded can be language dependent and impacted by a cultural environment. Thus, dedicated research efforts for languages other than English are required. In this paper, we conduct a systematic literature review to identify and compare existing bias detection methods for non-English word embeddings and language models. In an interdisciplinary team we examine the technical aspects, as well as the definitions of bias used by researchers in the field. Based on our findings, we outline a research plan for making bias detection in the field of NLP more inclusive for languages other than English.