Self-supervised learning using unlabeled speech with multiple types of speech disorder for disordered speech recognition

This paper investigates a training method of an automatic speech recognition (ASR) model for people with speech disorders. Because the characteristics of their speech differ significantly from those of the typical speech, in order to recognize the speech of a user with a disorder, the system needs to be trained with the user’s speech in advance. However, recording speech from people with disorders is a large burden for them, and therefore, it is difficult to collect a sufficient amount of speech for training. To address this issue, this study investigates the use of two types of speech as training data. The first type is unlabeled speech, which can be easily collected but lacks text labels (e.g., spontaneous speech in daily life). To utilize the unlabeled speech for training an ASR model, a self-supervised learning approach is employed. The second type involves utilizing speech data from individuals with different types of speech disorders. In our system, besides the user’s speech, the speech of individuals with the same type of disorder and even different types of disorders is also incorporated. Experimental results demonstrated that using unlabeled speech and speech from multiple types of disorders led to reduced recognition error rates.