Dialect speech recognition in small datasets: fine-tuning and enhancement strategies for Whisper models

With the promotion of Mandarin and the acceleration of the modernization process, China's dialect culture faces the risk of extinction, which is further aggravated by the lack of its digital resources. In this paper, we take a small-scale Qingyuan dialect dataset as the research object, and explore an effective method to improve the performance of speech recognition through data enhancement techniques. Using the Whisper model as the infrastructure, four sets of experiments are designed to compare and analyze the effects of data enhancement on different scale models (Whisper-tiny and Whisper-small). The experimental results show that data enhancement significantly improves the recognition performance of the models, in which Whisper-small outperforms Whisper-tiny in CER and WER metrics, verifying the synergistic effect of data enhancement and model size. The research in this paper provides an effective solution for small-scale dialect speech recognition, and at the same time provides technical support for the protection and inheritance of dialect culture.