Transformer-Based Embeddings for Greek Language Categorization

The Greek School Network (GSN) provides support to students, teachers, and school units in secondary education across Greece. Handling numerous user queries manually can be challenging, necessitating the development of an automated system for accurate categorization of these queries. This paper presents a comparative study of various transformer-based models for multi-class text categorization of Greek language queries submitted to the GSN helpdesk. We introduce a new experimental balanced dataset and extract vector representations from eleven transformer-based models. These representations are evaluated using ten classic machine learning classifiers. Our findings highlight the superior performance of the Multilingual E5 Text Embeddings model, particularly when paired with the extreme gradient-boosting classifier. This combination demonstrates a clear advantage in accurately categorizing user queries, paving the way for more efficient automated helpdesk systems.