Chinese Punctuation Restoration Based on Transformer Model

This paper explores the application of Chinese punctuation restoration technology utilizing the pretrained Transformer model XLM-RoBERTa-Base. With the increasing use of digital communication and development of automatic speech recognition, the absence of punctuation can lead to problems like misunderstandings, thus highlighting the importance of effective punctuation restoration technology. This paper focuses on Chinese punctuation restoration with data collected from the IWSLT2012 dataset, where punctuation signs are replaced with designated labels for preprocessing. The Transformer model is fine-tuned on this dataset, focusing on restoring punctuation tasks. After the training, the model's performance is evaluated using metrics including loss, accuracy, PR curves, and confusion matrices. The results indicate that, compared to existing models, the model outperforms in restoring Period and Question. However, the capacity of restoring Comma fails to be enhanced and remains at a moderate level. Also, the punctuation restoration of Chinese faces a performance drop compared to English but improvement compared to Bangla, with same model architecture. Over all, our findings demonstrate the applicability of pretrained Transformer models in Chinese punctuation restoration and suggest avenues for future improvements, particularly in enhancing comma restoration.