Optimizing Numerical Reasoning with Pre-trained Language Models: An Enhanced Algorithm

Currently, in the field of natural language processing, hybrid numerical reasoning tasks are more difficult to implement because they require models to have both reading comprehension and numerical reasoning capabilities. Traditional methods are not good at separating the generation of numerical operators from operands, which leads to poor performance when facing complex numerical calculations. The recently proposed ELASTIC model clearly separates the generation process of the two through a series of innovative mechanisms, laying a new foundation for solving complex and long numerical reasoning problems. However, the model still faces bottlenecks such as complex data preprocessing, increased resource requirements, limited domain transferability, and feature redundancy. To address these issues, we introduced a series of optimization techniques based on the ELASTIC framework. First, we use the data preprocessing procedure of enhanced ChatGPT, then strategically optimize the model parameters, and combine the Adapter and LoRA methods to improve the transfer learning effect, thereby comprehensively improving the model performance. Rigorous experiments on the FinQA dataset verify the effectiveness of these optimization methods, and the execution accuracy and program accuracy are improved by 1.25 and 1.7 percentage points respectively compared with the reconstructed ELASTIC model baseline, while achieving model parameter compression.