Veštačka inteligencija u nastavi stranih jezika - evaluacija pouzdanosti velikih jezičkih modela sa fokusom na srpskom jeziku kao stranom

In foreign language teaching, language competence implies, among other things, understanding dialectisms, phraseology, idioms, proverbs, and gnomes. The aim of this paper is to explore the possibilities of using artificial intelligence at higher levels of foreign language teaching, using Serbian as an exemplary foreign language. Three large language models (Claude, ChatGPT, and Gemini) were used for the purposes of this research. Three texts of a different level of difficulty, and written in different Serbian language dialects, were entered into the three language models, and the language models were asked to translate the texts into the standard Serbian language. The research sample consisted of 113 respondents - non-philologists, foreign language teachers, and teachers of Serbian as a mother tongue or a foreign language. An anonimous questionnaire, created on Google Forms, was the research instrument and it was distributed to the evaluators via email, i.e., to their channels and profiles on social networks. Based on a predefined list of criteria, the research participants had to evaluate the quality of the texts, without knowing that the texts were translated by artificial intelligence. Descriptive statistics techniques were used to process the obtained statistical data, as well as the one-way ANOVA with post-hoc tests (Bonferroni test). The research results indicate that Claude and ChatGPT can be considered reliable in translating texts written in dialects. When it comes to the lexical accuracy of the texts, the texts generated using these two tools were evaluated as very reliable, making it easier for students to understand dialects, phraseology, idioms, and proverbs in Serbian language. Based on the obtained results, the paper offers didactical implications for using artificial intelligence in foreign language teaching.