Evaluation of translations into Plain German produced by humans and MT systems including ChatGPT

In this paper, we present the results of a study evaluating intralingual machine translations of health information texts into Plain German. In our study, we compare machine-translated simplified texts with those simplified manually by human translators, as well as with the original, unsimplified texts. We compare the output of four different machine translations systems and assess the translation quality using various criteria, including translation errors, readability, and syntactic complexity. The study reveals that from the four analysed machine translation systems, ChatGPT performed worst. Our results also suggest that fine-tuning a model with task-specific and domain-specific data improves the translation quality.