When Whisper Listens to Aphasia: Advancing Robust Post-Stroke Speech Recognition

Despite recent advancements in Automatic Speech Recogni tion (ASR), its accuracy remains low for pathological speech, thereby limiting AI-based healthcare interventions in such set tings. This work addresses this challenge by fine-tuning Whis per, an ASR known for its ability to capture high-dimensional features in healthy speech. Using our comprehensive dataset of patients with stroke, we fine-tuned Whisper and significantly reduced Word Error Rate (WER), surpassing previous work on severe aphasia. To demonstrate its generalisability, we tested the model on a separate database, AphasiaBank, and observed a lower WER despite variations in dialect, linguistics, and test protocols. Our result on the AphasiaBank was superior to pre vious ASRs trained on this database, confirming the generalis ability of our approach. These outcomes not only address ASR limitations in impaired speech but also establish the foundations for standardised and versatile AI solutions for remote speech monitoring for timely diagnosis and intervention. Index Terms: Speech Recognition, Fine-tuning, Pathological Speech