Evaluating the extraction of Italian institutional terminology

Abstract In institutional settings, terminology management is essential to ensure efficient communication. Traditionally, this task has been carried out manually or through the use of corpus analysis tools. However, recent advances in generative artificial intelligence have opened new avenues for automating terminographic tasks. In this context, the use of generative models is proposed for the extraction of specialised terminology in academic institutions. Specifically, this study compares two approaches to terminology extraction. On the one hand, a corpus-based approach using the Sketch Engine tool and, on the other hand, an approach based on generative artificial intelligence. To this end, UniPDTerms was implemented — a chatbot designed with ChatGPT-4o specialised in institutional terminology of the University of Padua (Italy) and fed with an ad hoc corpus. The evaluation of both systems was performed using a reference list and analysing precision, recall, F-score and MRR metrics for each model. The results indicate that Sketch Engine and UniPDTerms performed at a comparable level under identical evaluation conditions. Although the two systems use different extraction mechanisms, their outputs produce similar results: Sketch Engine extracts relevant term candidates using frequency-based corpus analysis, whereas UniPDTerms draws on contextual and semantic relations. These results highlight the potential of incorporating generative artificial intelligence into terminographic workflows, offering new possibilities for improving efficiency and supporting end-users in terminology management.