Assessing AI-Powered Translation Quality: Insights from the Translation of the Farewell Sermon

Artificial Intelligence (AI) powered translation influxes in the translation industry. Yet, questions have arisen about the quality of AI-powered translation systems (AIPTSs). Therefore, this study aims to evaluate the translation quality gendered by ChatGPT to the Farewell Sermon. Using House's (2015) model of translation quality assessment (TQA), the study analyzed the Farwell Sermon focusing on text, register analysis of field, tenor mode and genre. The language/text analysis revealed that AIPTSs produced lexical and syntactic inaccuracies that hindered capturing the sermon's intended religious voice. In register, Field, the analysis pointed out that the AIPTSs captured the broad themes of the Farewell Sermon, but often missed subtleties in religious terminology. AIPTSs also adopted superficial literal translation of the sentence structures. In Tenor, the analysis revealed that the Prophet's role as a moral and spiritual guide was somewhat diluted in the translation. Likewise, AIPTSs struggled to maintain the Prophet's authoritative and compassionate stance. The AIPTSs maintained general coherence but faced challenges with cohesive devices such as conjunctions, pronouns, and references. Furthermore, the microanalysis revealed an amount of 81 overt translation errors that the AIPTSs committed while translating the Farwell Sermon into English, the most frequent were "creative translation" with 24 errors, "not translated" with 15 errors, "distortion of meaning" with 9 errors, "slight change in meaning" with 8 errors and "breach of the SL system" with 7 errors. "Significant change in meaning" and "cultural filtering" were less noticeable with 6 and 4 errors, respectively. Additionally, the analysis revealed that House's (2015) model is to some extent suitable for assessing the quality of AIPTSs, one type of error was technical and beyond the model focus though. The researchers called it "software intervention". Software interventions can either positively or negatively affect the translation depending on whether the error improves readability without distorting meaning or inaccuracies. The study recommends the importance of tuning the AIPTSs applications to prevent the inducing of technical systems.