Ensemble of Task-Specific Experts for Multi-Accent Speech Recognition

Despite recent advances in automatic speech recognition (ASR), accurately recognizing speech from speakers with varied accents remains a significant challenge. In this paper, we introduce an Ensemble of Task-Specific Experts (ETSE) approach to enhance multi-accent speech recognition capabilities. The proposed method trains experts specialized in different accents, allowing each expert to specialize in a particular accent while dynamically adjusting contributions based on the input speech. We evaluate our model on the CommonVoice corpus. Experiments demonstrate that our method not only significantly reduces word error rates (WER) on low-resource accents but also achieves a 7% improvement in performance over the fine-tuning baseline.