Empirical Evaluation of Prompting Strategies for Fact Verification Tasks

Commercial large language models (LLMs) such as GPT-3.5 have emerged as powerful tools for diverse natural language processing (NLP) tasks, yet concerns persist about their reliability in generating factual responses.This study 1 investigates the potential of commercial LLMs such as GPT-3.5 for fact verification, addressing three key research questions: (1) Can GPT-3.5 perform fact verification reliably?(2) How do different prompting strategies affect their performance?(3) What are the common errors they make with the most effective prompt?Using the benchmark FEVER 1.0 dataset, we designed and evaluated three prompts, with experiments conducted using GPT-3.5 as a representative commercial LLM.Our experiments demonstrate that GPT-3.5 achieves a Label Accuracy (LA) of over 72% with the best-performing prompt, significantly outperforming simpler prompts.A detailed error analysis reveals that approximately 70% of mistakes stem from logical reasoning and contextual misunderstandings.These findings suggest that carefully crafted prompts can substantially enhance the accuracy of LLMs, in fact verification tasks, highlighting their potential as supportive tools for applications in sensitive domains.