Comparing ChatGPT-4 and Human Translation of an Outcome Questionnaire: A Randomized, Double-Blinded Non-inferiority Study

Introduction This study aimed to investigate whether ChatGPT-4 could replace traditional forward-backward translation by humans when translating a questionnaire into another language through a non-inferiority design. Methods The questionnaire Abdominal Hernia-Q was translated from English to Danish twice using two different modalities: once with traditional forward-backward translations by human translators and once with ChatGPT-4, resulting in two parallel arms. Each version was randomly assigned to 10 participants who had previously undergone ventral hernia repair. All 20 participants had to evaluate the two translations randomly, both before and after face validation. The participants and the interviewer who conducted the face validation were blinded to the translation modalities. The participants were asked to evaluate readability, comprehension, appropriate use of language, and their overall opinion of the two translated questionnaires before and after face validation on a Likert scale of 1-5 . For each outcome, the non-inferiority limit was set to 1 point on the Likert scale. Results Both translations were evaluated by 20 participants before face validation and 18 participants after. Before face validation, relevant and significant differences were found for all outcomes favoring the human-translated version, and most participants accurately guessed the translation modality and preferred the human translation. After face validation, there were no significant differences between the assessments of the two versions of the questionnaire, their guesses were less accurate, and no significant difference in their preference was observed. Conclusion ChatGPT-4 may serve as a suitable substitute for traditional forward-backward translation when translating an outcome questionnaire, provided that the ChatGPT-4 translation is subjected to human quality control and is pretested through face validation before clinical use.