Handling of Out-of-Vocabulary Words in Statistical Machine Translation System

In machine translation Out of Vocabulary (OOV) word is an issue in low resource languages. OOV refers to those that are not present in the training data or cannot be translated using the corpus. Sometime these words are rare words in training data. Many techniques have been proposed by researchers to alleviate this issue in machine translation. In this paper, to obtain correct translations of target side OOV words by updating corpus for low resource Myanmar-Mon language pairs. The longest common substring method and Jaro Winkler similarity are used to extract the appropriate word for the unknown word from the phrase-based corpus. Mon language is Myanmar dialect and there are different Mon phrases with the same meaning as a Myanmar phrase. Thus, word approximation and phrase table updating are utilized to identify and substitute word to improve translation quality and alleviate OOV problem. Myanmar-Mon machine translation system are trained using phrase-based Statistical Machine Translation (SMT). Improvements are made in terms of word translation accuracy by combining with word approximation method.