Evaluation of the Quality of ChatGPT in Translating Texts on Indonesian War of Independence From Dutch to Indonesian
ChatGPT has been proven to have good translation capabilities. However, for texts in certain specific domains, the quality of the ChatGPT translation results still needs to be verified. This paper presents a preliminary work on the evaluation of the quality of ChatGPT in translating Dutch texts on the Indonesian War of Independence to Indonesian. We compare ChatGPT translation against Google Translate, using Bilingual Evaluation Understudy (BLEU) and Semantic Text Similarity (STS) scores. The scores were calculated between the provided human translation references and both machine translation outputs. ChatGPT has an average BLEU score 0.24 and an average STS score 0.80. Meanwhile, Google Translate has scores 0.25 and 0.79, respectively. This result shows that ChatGPT and Google Translate have comparable capability in translating the text under study.
Authors
- Dian Isnaeni Nurul Afra (ORCID: https://orcid.org/0000-0001-5128-9682)
- Mohammad Teduh Uliniansyah (ORCID: https://orcid.org/0000-0002-5224-3704)
- Tri Sampurno (ORCID: https://orcid.org/0009-0009-5274-3483)
- Topan Try Harmanda
Institutions
- National Research and Innovation Agency (ID)
Publication Details
- Published
- 2024-10-09
- DOI
- https://doi.org/10.1109/ic3ina64086.2024.10732746
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00