Research on the Construction Method of Tibetan Syllable Confusion Sets for Text Correction

Error confusion sets refer to collections of words or characters that, within the same group, are similar in spelling and grammar to the corresponding words or characters in the correct text, thereby easily causing confusion. Such confusion sets are commonly used in automatic text correction systems, where the system replaces potential errors with other candidates from the confusion set to enhance the accuracy of the correction. This paper introduces a method for constructing Tibetan syllable confusion sets aimed at text correction, and validates the confusion sets in experiments on Tibetan text correction. The Tibetan syllable confusion sets presented in this paper are composed of three distinct subsets based on the characteristics of Tibetan text: a confusion subset based on Tibetan syllable component information and edit distance, a confusion subset based on input key position distance, and a confusion subset based on Tibetan grammatical knowledge. These subsets are merged sequentially to form a comprehensive Tibetan syllable confusion set for text correction, totaling 59.2M in size. Experiments on public data with this confusion set have shown that the accuracy of the Bert+ confusion set correction method is 10.63% higher than that of a standalone Bert model, demonstrating the effectiveness of the Tibetan syl-lable confusion set in Tibetan text correction tasks.