Enhancing Financial Reasoning in Large Language Models: The Role of Gold Facts

Large language models (LLMs) require vast and high-quality training datasets, which are often difficult to annotate, particularly in domains such as finance. In finance, the datasets not only encompass large volumes of textual data but also involve intricate numerical reasoning, frequently presented in the form of tables. One significant challenge in this domain is the processing of exceptionally long financial statements and complex tables, which can hinder the reasoning capacity of LLMs and negatively impact their accuracy. To mitigate the adverse effects of lengthy text passages and complex tables, this study focuses on identifying and extracting key information (gold facts) that are essential for solving financial problems, with the aim of enhancing both reasoning quality and data efficiency. We investigate the disparities between various financial datasets and examine different model extraction techniques. Our experimental findings demonstrate that the precise extraction of these key facts from both textual and tabular data substantially improves the accuracy of LLMs-based question answering, while simultaneously reducing the amount of data necessary for effective model performance.