Assessing the accuracy of Chinese speech-to-text tools for Chinese as foreign language learners

Abstract This article examines the effectiveness of four Chinese Speech-to-Text tools in transcribing the speech of Chinese as a Foreign Language (CFL) learners across different ACTFL proficiency levels. The results indicate notable differences in transcription accuracy. Among the CSTT tools, ChatGPT 3.5 proves to be the most accurate, followed by WeChat and Baidu IME, while iOS IME shows the lowest performance. Except for iOS IME, these tools achieve 100% accuracy at the Distinguished and Superior levels, where speech closely approximates native fluency. ChatGPT 3.5 excels from Novice to Distinguished levels but occasionally overcorrects Novice-level CFL learners’ erroneous speech. WeChat performs robustly above the Novice level, while Baidu IME is best at the Advanced level and above. Conversely, iOS IME displays significant limitations at all levels. This study offers new perspectives on “good pronunciation” and the debate over handwriting versus typing Chinese characters for CFL learners.