Comparative Evaluation of ChatGPT, Google Translate, and UD Talk for Chinese-to-Japanese Translation in Cardiology and Pulmonology Outpatient Consultations: Prospective Observational Study

Background Language barriers between health care providers and patients can compromise communication quality, patient safety, and health care equity. When professional interpreter services are limited, particularly in outpatient settings, artificial intelligence–based translation tools may serve as supplementary communication aids. Objective This study compared 3 Chinese-to-Japanese translation tools (ChatGPT, Google Translate, and UD Talk) in terms of their performance in real-world cardiology and pulmonology outpatient consultations. Methods In this single-center prospective observational study, audio-recorded outpatient consultations between December 2024 and November 2025 were analyzed. Verbatim physician-patient dialogues were translated using the 3 systems. Selected dialogue exchanges were evaluated by professional medical interpreters for translation accuracy and by Japanese-speaking lay participants for translation satisfaction using a 6-point Likert scale. Similarity of translation outputs was assessed at the dialogue exchange level. Results A total of 20 outpatient consultations comprising 2450 dialogue exchanges were analyzed. ChatGPT had significantly higher translation accuracy and satisfaction scores than Google Translate and UD Talk, with median scores of 5.0 (IQR 4.0-5.0) for both outcomes compared with 2.0 (IQR 1.0-3.0) for Google Translate and UD Talk (P<.001), and maintained a consistently high performance in both specialties. Google Translate and UD Talk exhibited substantially higher similarity (2131/2450, 87%) to each other than to ChatGPT in translated outputs (119/2450, 4.9% to 122/2450, 5%, respectively; P<.001). Conclusions In Chinese-to-Japanese translations of outpatient medical consultations, ChatGPT demonstrated higher accuracy and user satisfaction than Google Translate and UD Talk. These findings indicate that artificial intelligence–assisted translation may support multilingual clinical communication when used as a complementary aid alongside professional interpreter services.