BERT-like Models for Automatic Morpheme Segmentation of the Russian Language

Current approaches to automatic morpheme segmentation for the Russian language rely on machine learning, primarily neural network methods.Among the architectures presented, the best results have been achieved using convolutional neural networks and LSTM networks.However, the quality of automatic annotation is far from ideal, especially when dealing with roots that were not present in the training dataset.In this work, we present a new approach to morpheme segmentation based on fine-tuning BERT-like models.Through comparisons using two morpheme dictionaries with different segmentation paradigms, we demonstrated the superiority of our approach over previous ones, including when working with unfamiliar roots.The best result was achieved by fine-tuning the RuRoBERTa-large model: when working with random words, the share of completely correct segmentations increased from 88.5-90.8% to 92.5-93.5%, and when working with unfamiliar roots, it improved from 70.5-72.6%to 74.9-77.2%.Error analysis of the model showed that root nests not encountered in the training dataset can be distributed into two groups during testing: "recognizable", meaning those for which more than 90% of the words are correctly analyzed, and "unknown", meaning those for which the proportion of correct segmentations is less than 10%.