A memory-aware speaker adaptation method for disordered speech recognition

Automatic speech recognition(ASR) is crucial for prompting effective communication in Patients with speech disorders. However, the high acoustic variability and data scarcity in disordered speech pose significant challenges to its recognition. Existing speech recognition methods struggle to address these challenges. Therefore, this paper proposes a memory-aware speaker adaptation method, enabling the ASR model to adapt to specific speakers while retaining common speaker features. Furthermore, this paper decouples speech representation into speaker information and semantic content. Accordingly, we develop two key modules: a memory-aware module, based on the read and write operations of a Neural Turing Machine, memorizes and retrieves speaker information; a content encoder, which includes a multi-level fusion module for aggregating semantic and acoustic features. Experimental results show that, compared to advanced speech recognition methods such as Transformer and time-delay neural networks(TDNN), our approach achieves 1.2% and 0.6% reduction in word error rate for aphasic speech and dementia speech, respectively.