CHARACTER-BASED BiLSTM FOR DIACRITIC RESTORATION IN SERBIAN

One of the most common irregularities in languages that contain letters with diacritics is the omission of diacritics. Their lack can lead to a misunderstanding and changes in the semantics of the text. Therefore, it is essential to restore diacritics automatically. This paper presents how contemporary deep-learning techniques can solve automatic diacritic restoration or diacritization problems without using powerful hardware with graphics processing units (GPUs). Training a neural network on a CPU is essential in an educational institution without proper new equipment, and this paper can encourage learning neural network programming in such conditions. We proposed a two-layer Bidirectional Long short-term memory (BiLSTM) sequential Neural Network for multiclass classification that learns to predict one of seven possible letters to replace a letter without the diacritic. The sequential nature of the text makes sequence networks efficient in Natural Language Processing and Understanding. The model was learned from Serbian text data. It is implemented using the character-based approach, a language-independent technique that can efficiently generate models for others, especially Slavic languages. The evaluation shows that all macro, micro, and weighted average metrics, such as precision, recall, and F1, achieved 98%. The main advantages of the proposed model are the easy and quick creation of a labeled dataset, not the very deep network, small vocabulary, small content window, a small number of fitting epochs, and easy manipulation of preprocessing and learning parameters to obtain the efficient and accurate model. The model is publicly available; it can be downloaded or tested on the corresponding website.