Using LLMs for measurement in diplomatic speeches: The evolving politics of UN sanctions in Security Council debates

Abstract UN sanctions are an important tool for maintaining international peace and have shaped the global security order since the end of the Cold War. However, given contemporary UN Security Council (UNSC) dynamics, new sanctions regimes have become rare, and policymakers are pessimistic about the future of the tool. We evaluate this pessimism by offering a long-term perspective, analyzing all UNSC members’ positions on UN-mandated and unilateral sanctions from 1992 to 2023, introducing the UNSC Sanctions Stance Dataset. We examine 5,984 sanctions-related Council speeches using a large language model for measurement, and validate this procedure with hand coding, descriptive analyses, and expert interviews. Our description of the construction process and guidance on using LLMs for measurement serve as a template for creating LLM-coded and manually and substantively verified datasets from IO speech data. The analysis focuses on Russia, China, and the Global South, and traces their positions in detail. We find that Russia turned from limited support of UN sanctions in the 2000s to total opposition in the mid-2010s, while China has persistently been opposed, offering at most begrudging tolerance. The P3 (France, UK, US) have consistently been supportive. Meanwhile, states from the Global South hold a distinct intermediate position, opposing unilateral sanctions while cautiously but consistently supporting UN-mandated sanctions. This finding cuts against an assumption of Global South opposition to sanctions in general.