LF Transformer: A novel lightweight network for machine translation

Machine translation automates the rendering of text from one natural language to another by utilizing computational algorithms, with the goal of producing translations that are semantically and syntactically accurate. Prevailing research recognizes neural machine translation (NMT), especially those using RNNs and Transformers, as highly effective. Yet, these models typically necessitate large sizes and an extensive number of parameters, escalading the computational load for both training and inference. Therefore, we a lightweight transformer model utilizing dual branches and a funnel structure, called LF Transformer. Each block employs long-distance attention mechanisms, dividing inputs into two parts for feature extraction by a global branch composed of a funnel-structured attention mechanism and a local branch composed of convolutions, ultimately combining the results of both parts. Comprehensive experiments are conducted on the Multi30k dataset, showing that LF Transformer achieves comparable performance to SOTA machine translation algorithms with fewer network parameters and computation costs. Also, LF Transformer performs better than SOTA lightweight transformer models for this task.