KET-GPT: A Modular Framework for Precision Knowledge Updates in Pretrained Language Models

Large language models (LLMs) have demonstrated outstanding performance across a variety of natural language processing tasks. However, the lack of precise control over knowledge injection, modification, and deletion presents a significant challenge, particularly for applications that require dynamic knowledge updates. In this paper, we propose KET-GPT, a knowledge editing framework built on GPT-2 XL, which introduces the Knowledge Editing Attention (KEA), Gated Knowledge Control Mechanism (GKCM), and Mixture of Experts (MoE) network. KET-GPT allows for efficient and targeted knowledge manipulation while maintaining linguistic fluency and coherence. Through extensive experiments, we demonstrate the effectiveness of KET-GPT in improving key performance metrics, including Edit Success (ES), Portability (PORT), Locality (LOC), and Fluency (FLUE). The results validate the significance of each component of the framework in achieving the desired knowledge editing capabilities. The proposed model offers an important step forward in the development of more flexible and adaptable LLMs, with potential applications in dynamic knowledge updates, factchecking, and personalized language model tuning.