PAIR-Stance: A Data-Centric Framework for Persian Social Media News–Post Stance Detection with Human–LLM Annotation and Targeted Augmentation

Abstract The rapid dissemination of information through social media has made reliable analysis of user reactions to news increasingly important. Stance detection provides a computational mechanism for identifying whether a social media post supports, contradicts, or is unrelated to a reference news item. However, Persian stance detection remains challenging because of limited reliable resources, linguistic variation in user-generated content, and severe class imbalance. In this study, we propose a data-centric framework for Persian news--post stance detection and construct a real-world dataset of 8,447 Persian news--post pairs collected from news sources and social media platforms. The instances are labeled as support, contradict, or unrelated through a hybrid annotation protocol combining large language model-based initial labeling, human verification, and final consistency review. Several modeling approaches are evaluated, including LaBSE, XLM-RoBERTa, and a Siamese LSTM, together with different imbalance-handling strategies. Among the baseline architectures, the Siamese LSTM achieves the highest macro-F1 score of 62%. Applying weighted cross-entropy substantially improves performance, producing the best overall result with a macro-F1 of 73% and a weighted F1-score of 92.67% on the test set. A complementary rationale-based model achieves an accuracy of 0.91, a macro-F1 of 0.73, and a weighted F1 of 0.90, whereas the proposition-based approach obtains a macro-F1 of 37%. Analysis of training and validation losses further indicates stable learning behavior with no clear evidence of severe overfitting. The results highlight the importance of annotation quality and imbalance-aware learning for real-world Persian stance detection.