Interjective Phrases in Film Scripts: Human and LLM Corpus Evaluation

This presentation provides a preliminary account of an ongoing doctoral research project that focuses on the annotation and pragmatic analysis of ritualized expressions (REs) in contemporary Spanish-language films. These expressions, including greetings, farewells, interjections, vocatives, polite formulas, and profanities, are pivotal for real-life communication, particularly in the context of Spanish as a Foreign Language (ELE). Preliminary findings suggest that traditional ELE manuals tend to underrepresent or oversimplify the contextual cues necessary for the appropriate use of REs, particularly in spontaneous spoken interaction (Leontaridi et al., 2023). The study adopts a quantitative and contrastive methodology that integrates manual lexicographic annotation with automated predictions from Large Language Models (LLMs), including ChatGPT, Claude, Gemini, Qwen, and Kimi. The study assesses the reliability of the annotation and the coverage of pragmatic traits in REs. The REs are extracted from an audiovisual corpus of Spanish-language films. They are then annotated using refined guidelines based on the Ritualized Expression Ontology (REO) and the concept of pragmatèmes, as defined by Mel’čuk (Mel’cuk, 1999, 2008, 2020, 2023) and Blanco (Blanco Escoda, 2013; Blanco Escoda & Mejri, 2018)). Particular emphasis is placed on interjective phrases, focusing on their prosodic, discursive, and interpersonal functions, in line with Halliday’s (Halliday, 1976) interpersonal metafunction. The study also considers diatopic variation across dialects. Each expression is analyzed according to its situational context (SIT), communicative function, and alignment with pragmatic categories such as modality, phaticity, or expressivity. The presentation compares the output of LLMs with expert lexicographer annotations, measuring inter-annotator agreement and exploring points of divergence. Preliminary results indicate that, while LLMs exhibit competence in surface-level classification, they frequently struggle with detecting deeper interpersonal or contextual features, such as register, formality, or diachronic nuance. These limitations underscore the need for pedagogically grounded descriptions of REs, especially in ELE teaching materials (Bernardi, 2012, 2013a, 2016). The methodology involves three stages: (1) extraction and categorization of REs from films, (2) annotation of pragmatic traits using a multilayered framework, and (3) comparative analysis of human-annotated and AI-generated labels. This triangulated approach aims to generate pedagogical recommendations for enhancing the treatment of REs in ELE curricula. This study contributes to the advancement of ELE pedagogy by offering an empirically grounded typology of REs and their pragmatic functions.