Evaluating a transparent and interpretable approach to stance detection using linguistic markers in social media data

Abstract Our study focuses on replicability, which entails researchers’ ability to achieve similar results to a prior study using identical methods but a different yet comparable dataset. We address the challenge of stance detection (determining whether a document is “favorable,” “against,” or “neutral” toward a target), building on prior research underscoring the value of linguistic markers as complementary features for sentiment detection that enable more accurate stance classification. We utilize the Stance in Replies and Quotes (SRQ) dataset, which contains annotated discussion-based responses. Employing a rule-based methodology that emphasizes linguistic features, we examine whether the classification accuracy remains within a similar error margin as observed in a previous study of another dataset. Consistency is a necessary condition for robustness and generalizability, ultimately enhancing trust in the methodology. The replication of the model and its adaptability to the new data context demonstrate that it is competitive compared to existing machine learning studies.