Synergizing Internal and External Knowledge: Prompt Engineering for Efficient and Effective Large Language Model Reasoning

Large language models (LLMs), such as ChatGPT, have demonstrated remarkable capability in question answering but face challenges when it comes to knowledge-based rea-soning, such as limited training data and hallucination. To address these challenges, integrating LLMs with knowledge graphs (KGs) has emerged as a promising solution. However, the cost associated with training and inference of LLMs is high. Our method integrates the Retrieval-Augmented Generation (RAG) paradigm, incorporating relevant information from KGs alongside the question to enhance LLMs' reasoning process without training. Moreover, we propose a novel concept of self-knowledge motivation to reduce the overhead of inference, which prompts LLMs to integrate retrieved information with their internal knowledge for reasoning before seeking additional queries to KGs. Experimental results showcase improvements in answer accuracy and a reduction in LLMs' API calls compared to the latest published state-of-the-art (SOTA) method employing an identical paradigm, underscoring the efficiency and effectiveness of our method.