Error analysis of translation outputs of the Siswati-English Linguistic Translation Tool

The development of machine translation (MT) systems for low-resourced languages in South Africa, such as Siswati, is crucial for bridging the linguistic divide between speakers of this language and English. Although MT systems for Siswati are available, there is a lack of studies that have assessed their translation outputs. This study evaluates the translation outputs of the Siswati-English Linguistic Translation Tool (SELTT), which translates from Siswati to English and vice versa. English texts from various secondary online open sources were collected and translated by the SELTT into Siswati. The automatically translated texts were then manually evaluated based on the Multidimensional Quality Metrics to identify errors in the outputs. The results show that, in some instances, the SELTT accurately translated texts into Siswati. However, translation errors such as omissions, mistranslations, spelling mistakes, incorrect capitalisation and untranslated words were identified in the outputs. This assessment provides necessary feedback to developers to refine and improve the performance of the SELTT.