Bilateral Treaties and Machine Translation: An Experiment in GPT-4 Translation Performance

Abstract Translation of international treaties has been one of the trending topics in the legal translation field (Biel 2006; 2017; Felici 2010; Prieto Ramos 2011; 2014; 2015; 2017; Prieto Ramos and Cerutti 2021; 2023; Šarčević 1997; Zhao and Cao 2013), for the complexity of it involving multiple languages and yet keeping the authenticity of each official language version. Despite the fact that the same principle echoes in bilateral treaties, very few studies have been conducted into the translation of this type of legal document (Khachatryan and Voskanyan 2022; Loiacono and Bertoli 2018), let alone in the Spanish-Chinese language pair.In the era of Artificial Intelligence (AI) and of large language models (LLMs) being the latest technological paradigm in translation practice, the authors intend to conduct an experimental study into the performance of AI with LLMs in the translation of bilateral treaties. To be precise, the authors used GPT-4 (version available in January 2024) to translate into Chinese the Spanish version of the Treaty between the Kingdom of Spain and the People’s Republic of China on Judicial Assistance in Civil and Commercial Matters; and compared the translated Chinese version with the official Chinese version of said treaty. In the comparative analysis, we found that GPT-4 presented several translation errors in all four aspects of TAUS typology, namely, language, accuracy, terminology and style. More specifically, the errors include a lack of legal register in the translation of vocabularies that exist both in ordinary as well as legal contexts; inaccuracy in the translation of terms and phrases; grammatical issues or un-naturalness shown in the translation of conditional subordinate clauses, gerund phrases and clauses in the passive voice in Spanish; failure in the translation of the official document title. These cases will be illustrated with examples and proposals for better translations.