A General Framework to Enhance Fine-tuning-based LLM Unlearning

Unlearning has been proposed to remove copyrighted and privacy-sensitive data from Large Language Models (LLMs).Existing approaches primarily rely on fine-tuning-based methods, which can be categorized into gradient ascent-based (GA-based) and suppressionbased methods.However, they often degrade model utility (the ability to respond to normal prompts).In this work, we aim to develop a general framework that enhances the utility of fine-tuning-based unlearning methods.To achieve this goal, we first investigate the common property between GA-based and suppression-based methods.We unveil that GA-based methods unlearn by distinguishing the target data (i.e., the data to be removed) and suppressing related generations-essentially the same strategy employed by suppressionbased methods.Inspired by this finding, we introduce Gated Representation UNlearning (GRUN) which has two components: a soft gate function for distinguishing target data and a suppression module using Representation Finetuning (ReFT) to adjust representations rather than model parameters.Experiments show that GRUN significantly improves the unlearning and utility.Meanwhile, it is general for finetuning-based methods, efficient and promising for sequential unlearning.Our code is available at github.com/renjie3/GRUN.