Enhancing Myanmar Sign Language Translation Through Transfer Learning, Self-training, and Error Correction

This paper investigates machine translation (MT) between Myanmar Sign Language (MSL) glosses and Myanmar Written Language (MWL). The MT models for both directions, MSL → MWL and MWL → MSL, are implemented by fine-tuning multilingual pre-trained language models, namely the mT5 and mBART-50 models. Subsequently, three techniques are employed to address the challenges posed to MT for these extremely low-resource languages. First, transfer learning is applied. The MT model is trained on a large-scale parallel corpus of American Sign Language and English and then re-trained using a small parallel corpus of MSL and MWL. Second, a self-training approach is employed. The translation pairs of MSL and MWL are created by translating sentences in a monolingual MWL corpus using the initially trained MT model. The synthetic and original MSL–MWL parallel corpora are then used to train the final MT model. Third, grammatical errors are corrected after the translation to enhance the quality of the translated sentences. An error correction model is obtained by fine-tuning the mT5 model using synthetic noisy data, which is automatically constructed by adding perturbations to the sentences in a monolingual MWL corpus. The experimental results show that each component of the proposed method contributes to improving the performance of MT between MSL and MWL. Furthermore, a comparative analysis is conducted on four token segmentation schemes, which are used as a preprocessing step in MT.