Evaluating and Enhancing Japanese Large Language Models for Genetic Counseling Support: Comparative Study of Domain Adaptation and the Development of an Expert-Evaluated Dataset
RAG demonstrated notable improvements across all evaluation metrics, suggesting potential for further enhancement through the expansion of RAG data. The expert-evaluated dataset developed in this study provides valuable insights for future optimization efforts. However, the ethical issues observed in JGCLLM responses underscore the critical need for ongoing refinement and thorough ethical evaluation before these systems can be implemented in health care settings.
Authors
- Shuntaro Yada (ORCID: https://orcid.org/0000-0002-6209-1054)
- Yusaku Urakawa (ORCID: https://orcid.org/0009-0006-2513-4250)
- Masayo Takahashi (ORCID: https://orcid.org/0000-0003-1836-6484)
- Eiji Aramaki (ORCID: https://orcid.org/0000-0003-0201-3609)
- Shoko Wakamiya (ORCID: https://orcid.org/0000-0002-9371-1340)
- Masae Manabe (ORCID: https://orcid.org/0000-0001-8018-4177)
- Shigeyuki Kan (ORCID: https://orcid.org/0000-0002-1889-5127)
- A. Maeda (ORCID: https://orcid.org/0009-0009-5935-1183)
- Takuya Fukushima
- Akiko Yoshida (ORCID: https://orcid.org/0009-0007-9914-5021)
Institutions
- Fujita Health University (JP)
- University of Tsukuba (JP)
- Kyoto University (JP)
- Kobe City Medical Center General Hospital (JP)
- Nara Institute of Science and Technology (JP)
Publication Details
- Journal
- JMIR Medical Informatics
- Published
- 2024-12-25
- DOI
- https://doi.org/10.2196/65047
- Citations
- 6
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.72