Comparison of the Performance of Transformer Text Summarization Models in Indonesian Language: PEGASUS and GPT-2

Manual text summarization is impractical and time-consuming. Therefore, the development of automatic text summarizers has become an interesting topic in Natural Language Processing (NLP). Pre-trained models are often developed from Transformer models, such as PEGASUS and GPT-2. This research aims to evaluate the performance of Transformer models trained with Indonesian language datasets for text summarization tasks. The advancement of these models will support the increased utilization of Transformers for developing Indonesian language text summarizers, both for research and industry purposes. Evaluation of the models was conducted by using precision, recall, and f1-score. The impact of using these 2 pre-trained models on the evaluation results was measured using One-Way Analysis of Variance (One-Way ANOVA). Evaluation of the models shows that the GPT-2 model achieves the highest mean of precision, recall, and f1-score values, which are 0.893375, 0.9094735, and 0.901187 respectively.