Zaznavanje stališč s pomočjo velikih jezikovnih modelov: validacija in prototipska analiza v nemško-govorečem subredditu r/de

This thesis investigates the efficacy of large language models (LLMs) for stance detection in German-language Reddit communities and explores their application in online public opinion research. The thesis evaluates various LLMs and prompting techniques on a custom-annotated dataset from the r/de subreddit. Experimental results demonstrate that commercial LLMs like Claude 3.5 Sonnet can perform stance detection on German Reddit comments with high accuracy (F1 scores >0.8) without fine-tuning. Open-source models also show promising results. Few-shot prompting with chain-of-thought reasoning consistently outperformed zero-shot approaches, while additional context harms performance across models. The thesis then applies the developed methodology prototypically to analyse public opinion trends regarding German military aid to Ukraine. Some correlation is observed between Reddit sentiment and broader public opinion polls, suggesting potential for using LLM-based stance detection on social media as a complementary tool to traditional polling methods. Limitations include the focus on a single subreddit and topic and reliance on a small manually-annotated dataset. Future research directions involve expanding to multiple communities and languages, investigating fine-tuning techniques, conducting longitudinal studies, and enhancing explainability.