Evaluating the Performance of Automated Speech Recognition Systems for Black Users

Automated Speech Recognition (ASR) systems understand and convert human speech into written text and even transform it into its own speech, as found in applications like Siri or Alexa. While these technologies have the potential to benefit the greater society, studies have shown they may not provide that benefit equitably across demographic groups. To better understand the extent to which ASR technologies support the diaspora of potential Black users, this systematic literature review covers existing research studies, performance evaluations, and technical reports related to ASR systems. Based on an analysis of articles discussing this topic, it was revealed that while efforts exist that aim to support Black users of ASR, there remain significant differences in the performance of ASR for Black and African American users. This could be because the primary focus of efforts thus far has been on the underrepresentation of African American Vernacular English (AAVE) in existing datasets and the need for exposure to diverse linguistic patterns during training. This review highlights important gaps in existing efforts to support the development of ASR technologies for the Black community, such as the lack of diverse data collection efforts.