Cross-Linguistic Examination of Gender Bias Large Language Models

Large language models (LLMs) have exploded in popularity recently, becoming widely used for various tasks. However, concerns persist regarding potential biases within these models. This study investigated gender bias in the popular LLMs such as GPT-3.5, GPT-4, Gemini, and Meta's Llama2. In this study, we focused on analyzing biases using ambiguous sentences across English, Malayalam, and Tamil languages. We evaluated the LLMs to see if they associate occupations with commonly held gender stereotypes, by using specific professions within our test sentences. We examined differences between the biases in the three languages in LLMs. Our findings suggested that strong gender biases exist in all the LLMs across the three languages. This study can be used to understand the biases in large language models and can be equipped to use AI in a way that mitigates bias.