<scp>Fin-Fact</scp> : A Benchmark Dataset for Multimodal Financial Fact-Checking and Explanation Generation

Misinformation poses significant risks to society, with the rise of cryptocurrency exchanges exemplifying the growing problem of financial misinformation. This issue is particularly troubling given the proliferation of false claims often paired with convincing yet misleading images. Despite the seriousness of this challenge, there is a notable absence of fact-checking datasets tailored to real-world financial claims. Additionally, given the high stakes of financial fact-checking, generating clear explanations for claim verdicts is essential to help decision-makers understand the reasoning behind these judgments. To address these challenges, we introduce Fin-Fact, a benchmark dataset comprising 3,369 financial claims. Each claim is annotated with a truthfulness label and a ruling statement, supported by both textual and visual evidence. We establish performance baselines for Fin-Fact using Gemini Pro and GPT-4. Our experimental results reveal that multimodal financial fact-checking remains a challenging task, even for cutting-edge generative models.