Undersökning av tvärspråklig generaliserbarhet för läsbarhetsbedömning med hjälp av en flerspråkig BERT-modell finjusterad på ett språk

Reading is a crucial part of language and educational development, but finding appropriate reading material suitable for an individual’s reading ability can be challenging. Consequently, there has been much research on automatic readability assessment using machine learning approaches. Recent research has shown that transformer-based models, typically using BERT (Bidirectional Encoder Representations from Transformers), outperform previous feature-based techniques in several languages. However, most research has focused on language-specific models and little on multilingual readability assessment. Since many smaller languages lack sufficient resources to train language-specific models, it is highly relevant to investigate whether a multilingual model fine-tuned on a single language can accurately predict reading levels of texts in other languages. In this thesis, a readability assessment model built using the multilingual BERT variant XLM-RoBERTa-Large, pre-trained on 100 languages and fine-tuned on Swedish literature, was evaluated. This was done by labeling Swedish books and their translated versions and comparing how similar the predicted labels are to the Swedish labels. The approach is based on two assumptions: that the Swedish labels are sufficiently accurate to be considered correct, and that the reading level of a book remains consistent across languages. The evaluation, which included 19 languages, showed that the model performed particularly well for languages that are linguistically close to Swedish, such as Norwegian and German, with accuracy declining for more distant languages such as Arabic, Urdu, and Pashto. This pattern held true in both a detailed nine-level readability assessment and a simplified three-level setup. While results varied, the model consistently demonstrated strong overall performance, often on par with language specific models, indicating its potential as a scalable solution for multilingual readability assessment, especially for languages with limited resources.