Fine-Tuning BART for Croatian Abstractive Text Summarization Using a Novel News Dataset

The aim of abstractive text summarization is to generate a condensed version of a text while preserving its essential information. Recently, language models based on transformer architecture have shown promising results in generating summaries. However, these models are not explicitly tailored for the Croatian language. Moreover, they are trained in a supervised setup using text summarization datasets, but there is still no available dataset specifically designed for Croatian summarization. This paper addresses both of these challenges by adapting the BART (Bidirectional and Auto-Regressive Transformers) model for Croatian abstractive text summarization, using a novel Croatian news summarization dataset curated from online news portals. Specifically, we fine-tune the pre-trained BART model for Croatian by utilizing this new dataset. We also discuss the challenges and pitfalls involved in constructing the dataset, particularly when using large language models as annotation experts. The performance of the fine-tuned BART model is evaluated using ROUGE metrics. The results demonstrate improvement in the quality of Croatian text summarization.