Evaluating BERT-Derived Models for Grammatical Error Detection Across Diverse Dataset Distributions

This paper presents a comprehensive evaluation of three language models: RoBERTa, SlovakBERT, and BERT-Multilingual, using datasets of varying sizes (ranging from 50,000 to 1,000,000 sentences) to assess their performance in natural language processing tasks in the Slovak language. The evaluation focuses on four critical metrics: accuracy, precision, recall, and F1 score across different dataset distributions (70:30, 50:50, and 30:70). The results demonstrate SlovakBERT's superior performance across nearly all metrics and distributions, particularly excelling in precision and F1 scores, which suggests its robustness and reliability in handling varied data distributions effectively. RoBERTa, while showing variability, performs well in precision under conditions of data imbalance but faces challenges in maintaining consistent accuracy. BERT-Multilingual, though slightly trailing behind, still delivers solid performance, especially in precision, making it a dependable model across varied conditions. The findings underscore the effectiveness of SlovakBERT in scenarios requiring high precision and reliability, and provide insights into the strengths and limitations of each model under different testing conditions.