Building Specialized Adversarial Examples for Improved Spelling Error Correction

Pre-trained language models based on deep neural networks have achieved superior performance in Chinese spelling correction. However, these models are heavily dependent on the size and quality of the training corpus and exhibit poor performance when faced with unknown errors. Inspired by adversarial learning, this paper focuses on Chinese character input scenarios and propose an effective attack strategy to generate more valuable training examples in these scenarios. This strategy jointly utilizes the attention mechanism and the logit output to determine the attack position and generates misspelling instances in a perturbed pinyin manner. The generated adversarial samples are then gradually added to the training set to train the model. Further, this paper explores the effect of the proportion of samples in the adversarial training process on model performance. The effectiveness of the proposed strategy are demonstrated on three different architectural baseline models.