Comparison of BERT-Based Classifiers for Detecting Grammatical Errors in Slovak Sentences

This paper presents a comparative analysis of clas-sifiers for detecting grammatical errors in Slovak sentences. We experiment with state-of-the-art transformer models, in-cluding the BERT multilingual base model, SlovakBERT, and XLM-RoBERTa, trained on datasets containing synthetically introduced grammatical errors. The datasets were created by corrupting words in various ways, simulating different types of grammatical errors such as incorrect letters, punctuation, and case sensitivity. These datasets range in size from 50,000 to 1,000,000 sentences, with varying ratios of correct-to-incorrect sentences. The models were fine-tuned on these datasets and evaluated using key metrics such as accuracy, precision, recall, and F1 score. Our results highlight the impact of dataset size, error type, and sentence distribution on the models' ability to detect grammatical errors. The findings provide valuable insights into the development of grammar-checking systems for the Slovak language.