Addressing Bias in Spoken Language Systems Used in the Development and Implementation of Automated Child Language‐Based Assessment

Abstract This article addresses bias in Spoken Language Systems (SLS) that involve both Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) and reports experiments to improve the performance of SLS for automated language and literacy‐related assessments with students who are under served in the U.S. educational system. We frame bias in SLS in terms of testing fairness and validity, stemming in part from the exclusion of sufficiently large training datasets in varieties of English other than General American English (GAE). We adopt an Interpretation/Use Argument approach to validity focused on clarity of constructs and scoring accuracy. While SLS use ASR to automatically transcribe students’ utterances, and apply NLP algorithms to ASR transcripts to measure students’ speech samples, it is well‐documented in studies with adults that ASR is typically more problematic for African American English (AAE) speakers than for other groups due to differences in prosody, pronunciation, word usage, and grammar. We utilized child speech and text corpora to improve algorithms that score oral task responses for child AAE speakers and, in some experiments, children with oral language and reading difficulties. Favorable results provide impetus and possible solutions for fair and inclusive assessments for diverse student groups in the future.