Caskow: Context-Aware Stance Detection Using External Knowledge-Augmented LLM

Stance detection in texts is an important task in natural language processing with diverse applications in political, economic and marketing. The research conducted in this area is categorized into traditional methods, deep learning methods and language model-based methods. However, to the best of our knowledge, none of these methods have effectively addressed context-aware stance detection. In this paper, we propose a method called CASKOW, which first collects context for a given text from Wikipedia as an external resource using information retrieval techniques and then employs the context alongside a large language model (LLM) to detect the stance. Experimental results show that CASKOW achieves a 7% improvement in F1-measure when the label “none” is included and a 10 % improvement when the label “none” is excluded, compared to models that do not utilize contextual information. Additionally, non-numerical interpretation of results reveals key insights, allowing for the identification of errors and challenges in stance detection datasets.