Comprehensive View of the Biases of Toxicity and Sentiment Analysis Methods Towards Utterances with African American English Expressions

Language is a dynamic aspect of our culture that changes when expressed in different technologies and/or communities. On the Internet, social networks have enabled the diffusion and evolution of different dialects, including African American English (AAE). However, this increased usage of different dialects is not without barriers. One particular barrier, the focus of this paper, is on how sentiment (Vader, TextBlob, and Flair) and toxicity (Google’s Perspective and models from the open-source Detoxify) scoring methods present biases towards utterances with AAE expressions. In particular, AI tools cannot understand the re-appropriation of the terms, leading to false positive scores and biases. Here, we study the bias of Toxicity and Sentiment Analysis models based on experiments performed on Web-and spoken English datasets.