International Journal for Advanced Research in Science & Technology

Large language models (LLMs) have shown significant potential in multilingual machine translation (MMT).This paper systematically explores the advantages and challenges of LLMs in this domain by addressing two key questions: 1) How effectively do LLMs translate a wide range of languages?2) What factors influence their translation performance?We assess popular LLMs, including XGLM, OPT, BLOOMZ, and ChatGPT, across 102 languages.Our empirical findings reveal that even the top-performing model, ChatGPT, falls short of the supervised baseline NLLB in 83.33% of translation tasks.Further analysis uncovers new patterns in LLM behavior during MMT.First, the semantics of prompts can be largely disregarded when using in-context examples, as LLMs maintain strong performance even with nonsensical prompts.Second, cross-lingual examples often provide superior task guidance for low-resource translations compared to same-language pairs.Lastly, we find that BLOOMZ's performance on the Flores-101 dataset may be overestimated, highlighting potential risks associated with using public datasets for evaluation.