Alignment Information via Optimal Transport and Pre-training for Neural Machine Translation

The demand for translation between different languages has increased rapidly as globalization has progressed. Although neural machine translation (NMT) has achieved excellent results through pre-training, existing models lack alignment information, which makes them unsuitable for specific domains. In this paper, a NMT model with alignment information via optimal transport (OT) and pre-training is proposed. First, the representation gap between different languages is narrowed by using alignment information via OT and pre-training (OTAP) to generate domain-specific data for information alignment, thereby learning richer semantic information. Second, a lightweight model, called the DR-Reformer, is introduced, which uses the Reformer as the backbone network and incorporates Dropout and Reduction layers to reduce model parameters and improve computational efficiency without sacrificing accuracy. Experiments conducted on the Chinese and English datasets of AI Challenger 2018 and WMT-17 show that our proposed algorithm outperforms existing algorithms.