A Deep Learning Approach for Yoruba Language Next Word Generation

Language models are important for modern neural network models that understand and create language. However, there is a significant problem when it comes to how well these models work with African languages due to the limited availability of training or test sets for low-resource languages. In this paper, will evaluate the performance of Byte-level Byte-pair Encoding (BBPE), which is more compact than character vocabulary and has no out-of-vocabulary tokens. It is implemented by a Bidirectional Long Short-Term Memory (Bi-LSTM) network for next-word generation using the Yoruba speech dataset from the open speech and language resources repository (OpenSLR 86). The experimental results show that the Bi-LSTM correctly predicted 5 reference texts out of 10 sequences, achieving a Bilingual Evaluation Understudy (BLEU) score of 1.0. It is likely that with additional training beyond 50 epochs, the model’s predictions will improve. The researcher anticipate that this study will pave the way for exploring other applications, such as video or text summarization, in the context of low-resource language modeling for African languages.