Enhanced Reverse Reasoning in LLMs for Further Research Directions Generation

Further research direction plays a pivotal role in the continuous innovation of scientific endeavors, serving as a critical driving force for disciplinary advancement and knowledge renewal. This study proposes an enhanced reverse reasoning method based on the masking mechanism in large language models (LLMs) to generate high-quality further research directions from scientific literature. We analyze the meta-knowledge required for generating further research directions, including research background, research questions, research methods, and research limitations. To extract meta-knowledge, we construct an instruction dataset and fine-tune the LLM using QLoRA. Experimental results show that, compared to the baseline model Llama3.1-8B, the fine-tuned model achieves significant improvements in F1 scores: +8.1% (82.6% vs. 90.7%) in research method extraction, +8.2% (82.0% vs. 90.2%) in research background extraction, +8.5% (81.4% vs. 89.9%) in research question extraction, and +8.9% (80.7% vs. 89.6%) in research limitation extraction. Based on the extracted meta-knowledge, we design three knowledge-driven masked prompt templates (masking research background, research questions, and research methods, respectively) to train LLMs’ reverse reasoning ability by occluding key information. Additionally, we propose an adaptive iterative feedback mechanism to dynamically identify and correct biases in generated content, continuously improving the model's reverse reasoning and generalization capabilities. Experimental results demonstrate that the fine-tuning method with knowledge-driven masking significantly enhances model performance, with ChatGLM3-13B achieving the most notable results: F1 scores of 91.7% (a 6.3% improvement) in research method reverse reasoning, 90.7% (a 6.3% improvement) in research question reasoning, and 89.7% (a 6.3% improvement) in research background reasoning. All fine-tuned models maintain F1 scores above 87.0%, with an average improvement of 6.0%. In the further research direction generation task, the ChatGLM3-13B model fine-tuned with the masking mechanism performs the best, achieving ROUGE-L and BLEU-4 scores of 0.845 and 0.823, respectively. Overall, the proposed method can be applied to further research direction generation tasks across various domains. Furthermore, by updating or replacing the types of meta-knowledge, the model can also serve as a theoretical foundation and paradigm for generating other types of scientific texts.