GENERATIVE AI AND OPEN-SOURCE DISTRIBUTION TO PUT EVALUATION TOOLS IN THE HANDS OF CLINICIANS AND PATIENTS: OSTOMY OUTPUT CONSISTENCY SCALE

Abstract BACKGROUND The Lincoln Ostomy Output Consistency Scale for jejunostomy, ileostomy and colostomy (LOOCS) is a medical instrument providing standardized stool descriptors for the ostomate community (Acad J Gastroenterol Hepatol 2021; 3:554). A limitation of many medical instruments is language accessibility. Language barriers in a clinical setting can hinder quality of care, lead to miscommunications, and decrease patient and physician satisfaction. Understanding these barriers, the Bristol Stool Form Scale has been translated in over 40 languages. Traditional translation involving human experts is often time-consuming and resource-intensive. Neural machine translation and generative artificial intelligence tools pose an opportunity to enhance translation efficiency. This project aimed to increase accessibility of the LOOCS. METHODS Using Ethnologue data, the top 100 most spoken languages were selected for translation, as well as several languages requested by bilingual speakers. The translation tools used included Google Translate, DeepL, ChatGPT, and MistralAI. Two separate translation tools were selected, labeled Tool A and Tool B, and four routes of forward and backward translation were used to determine the most effective and accurate route. The translation process included 7 steps. Step 1 involved preparing key phrases. Step 2 was forward translation using Tools A and B. Step 3 compared the two tool’s output. Step 4 was backward translation to English using Tools A and B on their respective forward translations. Step 5 was comparison to the original LOOCS. If the backward translation did not adequately represent the original LOOCS, steps 2-5 could be performed using different translation tools. Step 6 was selection of the most accurate translation based on likeness to the original content, fluency, and medical appropriateness. Step 7 (optional) entailed human validation. If verification of the AI translation by a human speaker is deemed invalid, steps 2-7 can be repeated using different translation tools. RESULTS The LOOCS was translated into 116 languages. Currently, 14 translations have been evaluated by bilingual speakers with 13 deemed accurate. According to the Urdu speaker, none of the AI tools produced an accurate Urdu translation. ChatGPT was deemed most accurate in 82 target language translations, Google Translate in 22, MistralAI in 7, and DeepL in 5. ChatGPT was able to translate 8 languages not able to be translated by any other tool, including Farsi and Karen. CONCLUSION This project demonstrated the utility and effectiveness of AI translation tools in a clinical healthcare setting. Though it appears AI translation tools can successfully translate the LOOCS, which uses simple and concise phrases, further research would be needed to determine their effectiveness in the translation of longer, more scientifically complex texts.