Artifacts for the Publication "Refusal Is Not an Option: Unlearning Safety Alignment of Large Language Models", USENIX Security 2025

Unlearning Safety Alignment Implementation of Refusal Is Not an Option: Unlearning Safety Alignment of Large Language Models, USENIX Security 2025. A more detailed and up-to-date document & codebase can be found in our GitHub repository. This repository is based on: Safe-RLHF RWKU MUSE Installation and Setup 1. Install AnacondaDownload and install Anaconda from here. 2. Create and Activate Python Environment conda env create -f environment.yaml conda activate unlearning If you wish to run IPython notebooks (required for Scenario II), also register the kernel: pip install ipykernel python -m ipykernel install --user --name unlearning Select the "unlearning" kernel in JupyterLab/Notebook or VSCode. 3. Set Your Hugging Face Token Insert your Hugging Face token in: hf_hub_token in LLaMA-Factory/src/llmtuner/hparams/model_args.py token in muse_bench/utils.py Set the environment variable for Hugging Face token: export HF_TOKEN=<hf_api_key> Scenario I Step 1: Construct Unlearning Dataset (Rejection Responses) Extract Model Responses Against AdvBench DatasetUse LLaMA-Guard to classify model responses by safety: cd harmfulness ./run_eval_harmful.sh Output: Results are stored in outs_llama/out_AdvBench. Process Rejection ResponsesExtract only safe responses to create the unlearning dataset: python rejection_responses_processing.py Output: Processed dataset at data/unlearning_data_AdvBench_3B.json. Step 2: Conduct Knowledge Unlearning All hyperparameters are set for training on two A100 80GB GPUs. Using DPO, NPO, or GAMove to script directory and run the desired method: cd LLaMA-Factory/scripts/full ./run_{method}.sh Output: Model/results: saves_3B/{method}_full_adv/${lr} Utility evaluation: results_3B/{method}_full_adv/${lr} Using TVFor the TV method: cd muse_bench/baselines/scripts ./run_tv.sh Output: Trained model in tv. Step 3: Evaluate Harmfulness Post-Unlearning Evaluate safety/harmfulness of the unlearned model: cd harmfulness ./run_eval_harmful.sh --eval_dataset {EVAL_DATASET} --model_name_or_path {MODEL_PATH} --output_dir {OUTPUT_DIR} Example: ./run_eval_harmful.sh --eval_dataset HEx-PHI \ --model_name_or_path ../LLaMA-Factory/scripts/full/saves_3B/dpo_full_adv/5e-6 \ --output_dir outs_llama/dpo EVAL_DATASET options: HEx-PHI, LLM-LAT_tot, or AdvBench Output: eval_results.json in {OUTPUT_DIR} For utility scores, see:LLaMA-Factory/scripts/full/results_3B/{method}_full_adv/{lr}/1_Stephen_King/{metric}.json(See Appendix B.4 in the paper for details.) For TV models: Set {MODEL_PATH} to the TV model directory. For utility, run: cd LLaMA-Factory/scripts/full ./run_eval_tv.sh Scenario II Step 1: Construct Unlearning Dataset (PII / FakeNews / Copyright) Generate Unlearning Dataset Using Two LLM AgentsIn data_merging directory, run the merging notebook for the target (PII, FN, CR): File: data_merging/Merging_{TARGET}.ipynb Output: data_merging/agents/GPT_results_score_{TARGET}.json API Cost Estimate: Merging with GPT-4o averages ~$10 per target (varies by size). Using GPT-4o-mini reduces this by >10x; just set the model to "gpt-4o-mini" in util.py. Filter Crafted Unlearning RequestsDownload the classifier for Harry Potter excerpts here.Run the filtering notebook: File: data_merging/Filtering_requests.ipynb Output: data_merging/agents/GPT_results_score_{TARGET}_passed.json Step 2: Conduct Knowledge Unlearning Use the filtered dataset; process is same as Scenario I. Update the unlearn_data field in your script (LLaMA-Factory/scripts/full/run_{method}.sh)(e.g., ../../../data_merging/agents/GPT_results_score_{TARGET}_passed.json) Run: cd LLaMA-Factory/scripts/full ./run_{method}.sh Step 3: Evaluate Harmfulness Post-Unlearning Assess safety/harmfulness as in Scenario I: cd harmfulness ./run_eval_harmful.sh --eval_dataset {EVAL_DATASET} --model_name_or_path {MODEL_PATH} --output_dir {OUTPUT_DIR} Repeat this process for each target (PII, FN, CR) as needed. Mitigation A BERT-based classifier for inspecting unlearning requests is provided in the defense directory. To train and evaluate, run: defense/BERT_classifier_adv.ipynb Supports BERT-based classification ($C_{enriched}$) and analysis of unlearning requests in modular steps. The inspected_data_path field can be set to the unlearning dataset you wish to analyze. Supplementary Materials For actual prompt templates and additional details, see the codebase and the following example files: figures/prompt1.png: Initial rewrite prompt for $A_{rewrite}$ figures/prompt2.png: Iterative rewrite prompt for $A_{rewrite}$ figures/prompt3.png: Evaluate prompt for $A_{eval}$ figures/prompt4.png: Initial rewrite prompt with synthetic PII figures/examples.png: Examples of merged synthetic data $D_{merged}^*$, $D_{reject}$, $D_{content}$ All image files are in the repository’s figures directory.