Named Entity Recognition for Pre-Modern Serbian Cyrillic: Using Transformer-Based Models and Generative Data Augmentation

Abstract In this paper, we evaluate the effectiveness of transformer-based models for Named Entity Recognition (NER) in pre-modern Serbian Cyrillic texts, with a particular focus on improving model generalization through generative data augmentation. We construct a domain-specific NER dataset, based on medieval Serbian charters and archival documents, annotated using the BIO tagging scheme, and extend the training corpus with synthetically generated examples. Two transformer-based language models, BERTić and XLM-RoBERTa, are fine-tuned and evaluated in both in-domain and out-of-domain settings. The results show that transformer-based models, particularly fine-tuned XLM-RoBERTa, achieve strong in-domain performance and substantially improve out-of-domain robustness compared with a previously developed CNN-based NER model. Old-Serb-XLM-RoBERTa trained with 30% synthetic data achieves the highest in-domain performance, with an overall F1-score of \((\approx)\) 0.92. The configuration trained with 100% synthetic data performs best on some individual out-of-domain documents, whereas the 50% configuration achieves the highest average weighted F1-score in out-of-domain evaluation and is therefore selected as the recommended model for broader use. These findings show that synthetic data augmentation can improve generalization, especially for low-resource historical NER, but its effectiveness depends on the model architecture, augmentation ratio, and the degree of domain shift. The recommended model, together with the annotated and augmented training dataset, is publicly available via Hugging Face, supporting further research on historical South Slavic languages.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-10275617/v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Named Entity Recognition for Pre-Modern Serbian Cyrillic: Using Transformer-Based Models and Generative Data Augmentation

Vladimir Polomac, Marija Đokić Petrović, Mihailo Popović
Research Square
Topic Modeling
preprint

Named Entity Recognition for Pre-Modern Serbian Cyrillic: Using Transformer-Based Models and Generative Data Augmentation

Vladimir Polomac, Marija Đokić Petrović, Mihailo Popović
preprint en

Abstract

Abstract In this paper, we evaluate the effectiveness of transformer-based models for Named Entity Recognition (NER) in pre-modern Serbian Cyrillic texts, with a particular focus on improving model generalization through generative data augmentation. We construct a domain-specific NER dataset, based on medieval Serbian charters and archival documents, annotated using the BIO tagging scheme, and extend the training corpus with synthetically generated examples. Two transformer-based language models, BERTić and XLM-RoBERTa, are fine-tuned and evaluated in both in-domain and out-of-domain settings. The results show that transformer-based models, particularly fine-tuned XLM-RoBERTa, achieve strong in-domain performance and substantially improve out-of-domain robustness compared with a previously developed CNN-based NER model. Old-Serb-XLM-RoBERTa trained with 30% synthetic data achieves the highest in-domain performance, with an overall F1-score of \((\approx)\) 0.92. The configuration trained with 100% synthetic data performs best on some individual out-of-domain documents, whereas the 50% configuration achieves the highest average weighted F1-score in out-of-domain evaluation and is therefore selected as the recommended model for broader use. These findings show that synthetic data augmentation can improve generalization, especially for low-resource historical NER, but its effectiveness depends on the model architecture, augmentation ratio, and the degree of domain shift. The recommended model, together with the annotated and augmented training dataset, is publicly available via Hugging Face, supporting further research on historical South Slavic languages.

Research Square
Austrian Academy of Sciences (AT), University of Kragujevac (RS), Union University (US), Union University (RS)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon