Expanding the UNSC Conflicts Corpus by Incorporating Domain Expert Annotations and LLM Experiments

In this work we expand the UN Security Council Conflicts corpus (UNSCon) (Zaczynska et al., 2024) on verbal disputes in diplomatic speeches in English.By including annotations of a UNSC expert, we target the problem of annotating verbal conflicts in a domain with its own culture and rules.On the one hand, we aim to catch all conflicts detected by political domain experts which as a result will be interpretable only by people with advanced political science backgrounds.On the other hand, we target linguistically marked verbalisations that are domain-independent and potentially easier to detect for language models.This balancing act resulted in a refined annotation scheme, and we re-annotate and expand the corpus size by 40% by including new debates.We perform a pilot study using a Large Language Model to include lexical markers of negative evaluation within the conflict spans, which until now were not annotated separately.Classification experiments on the conflict labels in the corpus using Transformer models demonstrate that models trained on the political domain improve the results.