Knowledge-Based Transformer: Enhancing Neural Machine Translation Through Knowledge Base Integration

The Transformer architecture, introduced since 2017, has demonstrated its power in machine translation tasks. However, significant challenges persist, notably in accurately understanding and converting meaning between languages. Out-of-vocabulary words, especially rare entities and terminological expressions in datasets, are the main cause of this inefficiency. We propose two methods integrating knowledge bases into the Transformer model to generate more accurate translations for entities and terms, thus improving overall translation quality. Specifically, these methods involve linking representation vectors for entities in the knowledge base to internal vectors generated within the Transformer architecture or directly replacing internal vectors with those of corresponding entities in the knowledge base. Our methods have shown promising results on language pairs from the IWSLT dataset, evaluated using four metrics: BLEU, TER, METEOR, and GLEU. Our source code is available on GitHub at https://github.com/VTaPo/KnowledgeBaseTransformer.