When Languages Collide: The Challenges and Progress of Speech Recognition in a Code-Switching World

Our world is becoming a multilingual world, and so is the manner in which we communicate. Code-switching—alternating among two or more languages within a conversation—is an integral facet of communication for millions. But for machines, particularly speech recognition software, this dynamism is a difficult one to crack. The majority of Automatic Speech Recognition (ASR) technologies are built around monolingual speech. In this research, we examine how ASR systems fare when confronted with code-switched speech, in which boundaries between languages shift and slide. We look at both the technical challenges and human linguistic behaviour that complicate this task, and also assess how much progress has been made by current approaches such as Whisper and wav2vec 2.0. We end with consideration of what is yet to be accomplished in making machines linguistically attuned and competent in our highly diverse world