Wav2Vn: Scalable Unsupervised Pre-training and Adaptation for Multi-Accent Vietnamese ASR

Automatic speech recognition (ASR) systems have seen significant advancements in recent years, driven by deep learning and large-scale pre-training methods. However, achieving high accuracy across multiple accents remains a challenging task, especially for languages with rich phonetic diversity like Vietnamese. The Vietnamese language encompasses a wide range of regional accents, each with unique phonetic characteristics that can hinder the performance of conventional ASR. To address this issue, we present a Vietnamese language processing model called Wav2Vn as a novel approach to scalable unsupervised pretraining and adaptation for ASR targeting multiple Vietnamese accents. Our method leverages the power of the Mixture of Experts (MoE) to effectively handle the diverse phonetic variations across different Vietnamese accents. Our results demonstrate a significant performance improvement in terms of word error rate (WER) and robustness to accent variability compared to the current state-of-the-art PhoWhisper.