Novel Sequential Knowledge Batch Editing via Retrieving Padding Right Embedding

Sequential Batch Editing is an effective method for updating knowledge in knowledge editing. Most existing methods require higher GPU hardware resources, occupy more GPU memory, and have longer training times. To address the above challenges, we propose an efficient and novel strategy called Retrieving Padding Right Embedding (RPRE). Our RPRE strategy can further improve the accuracy of knowledge retrieval by eliminating the interference of pad embeddings on knowledge. Specifically, our strategy is based on KVAttn implementation, which can extend this method from sequential editing to sequential batch editing, thereby improving training speed and overall model performance. KVAttn is mainly responsible for providing training techniques, knowledge storage, and knowledge retrieval techniques. Our RPRE is responsible for reducing the interference caused by pad embeddings in the knowledge storage and optimization process, further improving the accuracy of retrieval. The experimental results show that our method supports training large batch sizes and has superior editing and generalization abilities.