DIVERSE: A Dataset of YouTube Video Comment Stances with a Data Programming Model

Public opinion of military organizations plays a crucial role in their ability to recruit talented individuals. As recruitment increasingly extends into digital spaces like social media, assessing the stance of social media users toward online military content becomes essential. However, there is a notable lack of data for analyzing opinions on military recruitment efforts online, compounded by challenges in stance labeling, which is key to understanding public perceptions. Despite the importance of stance analysis for effective online military recruitment, generating human-annotated, in-domain stance labels is resource-intensive. In this paper, we address both the challenges of stance labeling and the scarcity of data on public opinions of online military recruitment by introducing and releasing the DIVERSE dataset<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>. This dataset comprises comments from videos on the U.S. Army’s official YouTube channel. We employed a state-of-the-art weak supervision approach, leveraging large language models to label the stance of each comment toward its respective video and the U.S. Army. Our findings indicate that the U.S. Army’s videos began attracting a substantial number of comments post-2021, with a generally balanced stance distribution among supportive, oppositional, and neutral comments, though with a slight skew towards oppositional over supportive comments.