Pragmatics understanding benchmark for Czech, Slovenian and Croatian PragMega CzeSloCro

PragMega CzeSloCro is a translation and adaptation of a section of the PragMega dataset (Floyd et al., 2026) into Czech, Slovenian and Croatian. The original dataset was manually crafted by psychologists and aimed at discovering whether "pragmatic inferencing" is a result of a single cognitive skill or, on the contrary, of different dissociable skills depending on the type of phenomena encountered. A Slovenian version of the dataset was created first, for which we selected three tasks: Irony, Metaphor, and Humour. These consist of 50, 30, and 25 examples, respectively, or 105 examples in total. The Slovenian benchmark is described in Brglez & Vintar (2026) and is implemented as SloPragMega on the SloBench (https://slobench.cjvt.si) platform. The dataset was then translated into Croatian and Czech by students of Digital Linguistics within a student project, then thoroughly revised by two professional linguists. Due to the highly nuanced and culturally specific nature of the dataset, some tasks were completely rewritten or replaced by more naturally sounding examples in the respective language. For each language, the dataset is divided into 3 subfolders (Metaphor, Irony, Humor), which contain the following files: - stim.csv: The actual localised tasks with possible answers, - stim_en.csv: The original English tasks with possible answers, - stimOrder.csv: Template to create a randomized test with the order of questions and answers reshuffled, - keys.csv: Solutions for the shuffled tasks. References: Brglez, M., & Vintar, S. (2026). From Polyester Girlfriends to Blind Mice: Creating the First Pragmatics Understanding Benchmarks for Slovene. In The Fourth Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL 2026) (pp. 44–54). European Language Resources Association (ELRA). https://doi.org/10.63317/4bpncy453r9k Floyd, S., Gibson, E., Fedorenko, E., & Poliak, M. (2026, January 14). PragMega. https://doi.org/10.17605/OSF.IO/DPGE6