A Comparative Study on the Translation Quality of Chinese Diplomatic Discourse by NMT and LLMs Based on Multidimensional Quality Metrics

Chinese diplomatic discourse plays a crucial role in articulating China’s position and enhancing its influence in global forums. However, machine translation (MT) often struggles with culturally nuanced and abstract expressions, highlighting the need to compare various advanced MT tools. This study assesses and compares the translation quality of Neural Machine Translation (NMT) systems and Large Language Models (LLMs) in translating Chinese diplomatic texts, focusing on the 2025 China-US tariff statements by China’s Foreign Ministry Spokesperson Lin Jian, with <i>China Daily</i>’s official English versions serving as references. Four NMT tools (Niutrans, Youdao, Google, DeepL) and four LLMs (DeepSeek, Ernie-4.5, ChatGPT-4.0, Gemini) were examined. Using the Multidimensional Quality Metrics (MQM) framework, the study evaluated translations, especially for phrases like “奉陪到底” (fight to the end) and “得道多助,失道寡助” (A just cause enjoys abundant support while an unjust one finds little). Results show that LLMs outperform NMTs: 50% of LLMs (DeepSeek, Ernie-4.5) accurately translated both phrases, while only 25% of NMTs (Google) did so for “奉陪到底,” and none for “得道多助,失道寡助.” Both systems faced issues such as undertranslation, omission, and a lack of diplomatic formality. The findings suggest that LLMs have greater potential to handle cultural nuances and abstract content in diplomatic texts, providing insights for enhancing domain-specific MT training and striking a balance between accuracy and acceptability in conveying Chinese diplomatic messages.