JSocialFact: a Misinformation dataset from Social Media for Benchmarking LLM Safety

The emergence of large language models (LLM) has given rise to a growing concern regarding the generation and dissemination of inaccurate information through these technologies. Addressing this issue requires a benchmark for the safety of LLM for Japanese. However, existing benchmarks are limited in that fail to adequately incorporate the unique falsehoods and erroneous information that are actively circulating on social media in Japan. This study proposes JSocialFact, a benchmark for evaluating the safety of LLM based on misleading information in Japan. The benchmark is created by manually annotating the data extracted from X posts and community notes capturing a wide range of misleading, false, or malicious information currently circulating on social media. Both manual and automatic evaluations using GPT-4 revealed discrepancies in how models handle harmful content. Although GPT-4’s evaluation showed some correlation with human judgment, notable discrepancies were observed, with GPT-4 frequently assigning higher safety scores than human evaluators. JSocialFact is the first dataset constructed from actual social media logs in Japanese, specifically designed to evaluate the safety of LLM outputs in addressing misinformation.