MCL-MT: Multi-Level Contrastive Learning for Many-to-many Multilingual Neural Machine Translation

Currently, mainstream multilingual translation models are trained mainly on English- related language pairs. These systems usually perform well on the English-related direc- tions, known as supervised directions, while the translation performance on non-English directions (zero-resource directions) is weak. Therefore, a method called mRASP2 has been proposed which integrates monolingual corpus and bilingual corpus under a unified training framework through contrastive learning and alignment enhancement methods, so that it can make full use of the corpus, learn bet- ter language-independent representations, and thus improve the performance of multilingual translation. In this paper, we propose a method to train an NMT model. Unlike the sentence- level alignment used in most previous stud- ies, this paper uses MCL-MT to explicitly in- tegrate word-level information of each pair of parallel sentences into contrastive learning. English-centric translation directions show su- perior performance with MCL-MT’s method- ology in comparison to the pretrained and fine-tuned model mBART. On non-English translations, MCL-MT offers an average 10+ BLEU score increase when compared to the multilingual Transformer baseline.