Research on morphological analysis of derived low-resource languages

In derived languages, word formation mainly depends on the combination of stems and affixes. Uyghur and Kazakh are typical derived languages. Sentences are composed of independent words. These words can be presented in two forms: One is the form of stem, which does not have any additional parts. The other is the form of stem with affixes added, which appears in sentences in most cases and directly affects the efficiency of text processing. In response to the above problems, this paper proposes a neural network stemming model for derived languages. This model fully integrates the features between sentence context and characters. The vector representation of the text is superimposed on the hidden layer information of the forward LSTM sequence and the hidden layer information of the reverse LSTM unit sequence, so that the bidirectional contextual semantic features of the sequence can be obtained. In the attention layer, the output of the BiLSTM layer is used as its input, and its input is semantically encoded, thereby improving the efficiency of word stem extraction. In order to significantly improve the effect of stemming, the relationship between the outputs is measured and normalized to obtain the global optimal solution. In order to verify the effectiveness of the model in this article, it was tested on the data set through different iteration times and methods, and a relatively good stem extraction effect was obtained, which laid the foundation for subsequent text processing research.