Developing a Robust Mispronunciation Detection by Data Augmentation Based on Automatic Phone Annotation

Limited non-native English speech data and accurate pro- nunciation labels results in data sparsity in Mispronunciation Detection and Diagnosis (MDD), hindering its performance. The purpose of this study is to investigate the impact of data augmentation using automatic phone annotation on the per- formance of MDD. We proposed to utilize Self-Supervised Learning (SSL)-based phone recognizers and Recognizer Output Voting Error Reduction(ROVER) to automatically generate phone annotations of the ESLTTS dataset, a corpus of non-native English speech that lacks phone-level annotations. We then evaluate the performance of MDD tasks. The experimental results indicate that the proposed methodology yields improvements in both Phone Error Rate (PER) and F1 score.