Optimization and Practice of Long Text Foreign Language Translation Algorithm Based on Transformer-XL Architecture

When processing long text foreign language translation, the traditional Transformer model faces computing resource and memory limitations when processing long sequences, which affects translation efficiency and quality. This study adopts a long text foreign language translation algorithm optimization method based on the Transformer-XL architecture, and expands the context length of the Transformer model by applying a segmented recurrence mechanism and relative position encoding, enabling it to process long text sequences more effectively. During the training process, segmented pre-training and autoregressive training strategies are adopted. A mixed expert strategy is also used to increase the capacity of the model, combining multiple expert networks together, each of which focuses on a specific part of the sequence. Transformer-XL outperforms Vanilla Transformer in BLEU (Bilingual Evaluation Understudy) scores; Transformer-XL’s TER (Translation Edit Rate) scores are between 0.10% and 0.79%, and its METEOR (Metric for Evaluation of Translation with Explicit ORdering) scores are also excellent, while its inference latency is lower than that of Vanilla Transformer. Ablation experiments further verify the important contributions of relative position encoding and dynamic caching to model performance.