acceptodds
Under review as a conference paper at ICLR 2027

LEAF-OPD: Learning to Forget from Privileged Contexts via On-Policy Distillation

Abstract

LLM unlearning aims to erase targeted knowledge from large language models while preserving their utility. However, many existing methods focus on suppressing target knowledge without explicitly teaching the model how to respond to queries about it. We propose LEAF-OPD (LEArning to Forget from Privileged Contexts via On-Policy Distillation), an LLM unlearning algorithm in which a frozen teacher policy provides token-level supervision on responses generated by the student policy. On forget data, the teacher policy uses privileged contexts (additional instructions not to reveal knowledge about forget targets) to guide forgetting. On retain data, the same frozen teacher policy provides supervision without privileged context, encouraging the student to preserve the teacher’s responses and thus maintain model utility. The student policy operates without privileged context in both cases. Instead of simply pushing the model away from the forget data distribution, LEAF-OPD explicitly teaches the student how to behave when encountering knowledge that should be forgotten. A key challenge is that the teacher usually provides effective supervision only at the first few tokens along the student's trajectories, with little effective supervision at subsequent tokens. To characterize the teacher's supervision strength, we introduce Supervision Signal (SS), which measures the strength of corrective supervision over a specified token interval. We evaluate SS over the entire trajectory (SS₁) and its second half (SS₀.₅) to assess the overall strength and late-stage persistence of corrective supervision, respectively. When supervision is weak, LEAF-OPD applies lightweight SFT to strengthen the teacher's context-conditioned refusal behavior. This can substantially increase both SS₁ and SS₀.₅, transforming a bad teacher into a good teacher. We evaluate LEAF-OPD on three widely used LLM unlearning benchmarks: TOFU, RWKU, and MUSE-News. LEAF-OPD achieves the best performance regarding Forget Quality, Model Utility, and Forget ROUGE-L on TOFU, while maintaining competitive retain performance. On RWKU, LEAF-OPD outperforms the second-best method by 34% in Forget Quality while maintaining competitive Retain Quality. These results demonstrate that privileged-context supervision with on-policy token-level distillation provides an effective approach to LLM unlearning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
OPSD, LLM Unlearn, Responsible AI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xwkvOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper presents an LLM unlearning method based on the technique of on-policy distillation, where the student model distills from a teacher model with extra refusal instructions called privileged context. To validate whether the teacher model with privileged context is capable of providing supervision, the authors introduce a supervision signal to quantify it. When the teacher model is not good enough, the authors apply further supervised fine-tuning to strengthen the refusal behavior.

Critical strengths
  • The proposed unlearning method of utilizing on-policy distillation on the model with privileged context is interesting and novel to me. The unlearned model is therefore able to learn what it should answer given forget queries.
  • The introduction of a supervision signal to quantify teacher supervision capabilities is interesting.
  • The experiments cover a wide range of baseline unlearning methods and benchmark datasets.
Critical weaknesses
  • While the authors criticize other training-free methods in the introduction for not being able to prove "whether the target knowledge is truly removed rather than merely suppressed at inference time", the proposed on-policy distillation-based method suffers from the same problem, in my opinion. This might be my major concern with this paper. The student is trained to reproduce the refusal behavior, but the factual content may still remain in the model and can still be recovered. The authors also did not show anywhere that the target data has been truly removed from the model.
  • The authors determine that the teacher model is considered good if the supervision signal passes 1. This seems to be a bit arbitrary. Can the authors provide more support or analysis on the correlation between the supervision signal and the unlearning performance?
  • While the authors claim that they seek to avoid the high sampling costs of RLVR-based methods, they have not provided any comparison of the efficiency between their proposed method and other baseline methods.
  • More support/explanations are needed for the adoption of the on-policy distillation technique. For example, how does on-policy distillation compare to directly applying SFT to strengthen refusal behavior, like in teacher model lightweight SFT stage 1? How much performance gain does on-policy trajectory provide compared to offline distillation?
Questions for authors
  • Can the authors add more supporting evidence for the true removal of data from the model?
  • Will the proposed method lead to over-refusals on other retain/unseen queries?
  • Can the authors consider adversarial settings such as relearning? How quickly does the OPD unlearned model relearn the forget responses?
6 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer xwkv

We thank you for your time, effort, and valuable comments; below are our responses to your comments.


Weakness 1. Clarify whether LEAF-OPD truly removes target knowledge rather than merely teaching refusal behavior, given the same concern raised about training-free methods.

We thank the reviewer for raising this important concern. Training-free methods do not update model parameters and primarily operate at inference time. In contrast, methods that update model weights, including LEAF-OPD, can modify the model’s internal knowledge representations and how that knowledge is used. Specifically, LEAF-OPD distills the teacher policy’s ability to distinguish between forget and retain data, refuse requests concerning forget content, and answer requests concerning retain content into the student policy. Thus, the student learns to selectively handle forget and retain requests, rather than simply learning to refuse. This idea is similar to RULE [2], but LEAF-OPD requires only one rollout per prompt, resulting in lower sampling costs than RLVR methods that require multiple rollouts.

Determining how and to what extent each unlearning method modifies the model’s knowledge to achieve genuine forgetting is an interesting research direction that we plan to explore in future work.


Weakness 2. Justify the SS threshold of 1 and analyze the relationship between supervision signal and unlearning performance.

Our criterion considers both the overall strength and the persistence of supervision in the second half of the trajectory, rather than only the refusal tendency at the first token. By definition, if corrective supervision is concentrated primarily at the first token, its contribution to the trajectory-averaged SS diminishes as the trajectory length increases, while the second-half SS remains close to zero. Conversely, if the teacher consistently provides strong corrective supervision throughout the trajectory, both metrics remain high. Figure 4 supports this interpretation: at both the individual-sample and dataset levels, the SS values are consistent with our characterization of good and bad teachers.

Regarding the relationship between supervision signal and unlearning performance, we examine this in Section 6 (Ablation Study). We evaluate a bad teacher under different configurations, including forward and reverse KL and first-token-only supervision, to assess the importance of a good teacher for LEAF-OPD.


Weakness 3. Compare efficiency with baseline methods to substantiate the claimed reduction in sampling costs relative to RLVR-based approaches.

Thank you for pointing this out. We will provide an efficiency comparison between LEAF-OPD and the baselines below and include it in the revised manuscript (ablation 5, section 6). Runtime comparison on Qwen3-8B.

MethodRuntime (min)MethodRuntime (min)
GA5LLMU7
GD6NPO8
KL9FLAT10
PO9LEAF-OPD20
DPO8

Weakness 4. Demonstrate the benefits of on-policy distillation through comparisons with direct refusal SFT and offline distillation.

Please refer to Response to Common Concerns Raised by Reviewers - 1.The tiny SFT on stage1 - b.The effect of tiny SFT on performance. We will also add this analysis to Ablation 6 in Section 6 (Ablation Study).


Weakness 5. Provide additional evidence that target data have truly been removed from the model.

Please refer to our response to Weakness 1.


Weakness 6. Assess whether LEAF-OPD causes over-refusal on retain and unseen queries.

The metrics in the TOFU benchmark [1], particularly model utility (MU), already account for this concern. LEAF-OPD’s strong MU performance on TOFU suggests that it does not exhibit substantial over-refusal on unrelated questions.


Weakness 7. Evaluate robustness to relearning attacks and how quickly the unlearned model relearns the forget responses.

This is a very interesting question. In this work, we primarily focus on LEAF-OPD’s unlearning quality and its preservation of model utility through a frozen teacher. Adversarial settings, including relearning attacks, will be a key focus of our future research.


We look forward to your response and would be happy to discuss any aspect of the paper with you.

[1] TOFU: A Task of Fictitious Unlearning for LLMs, COLM2024

[2] RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimality, nips2025

The authors have posted their responses to your comments. Could you please take a look and let us know whether your concerns have been addressed? We encourage active discussion between reviewers and authors during this period.

Thank you for your response. I appreciate the additional explanations, which should assist in the general understanding of this paper. However, my major concern remains: although the student model can learn to refuse requests concerning forget content, such refusal simply avoids providing forget content in reponses, but the forget knowledge still remains in the model. This may contradict what the authors define: "LLM unlearning aims to erase targeted knowledge from large language models". For example, the forget knowledge can be easily leaked or recovered through other means, such as membership inference or relearning attacks (as mentioned in question 3). I hope the authors can futher address my concerns.

Reviewer_W7M4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies LLM unlearning, aiming to suppress the reproduction of targeted knowledge while preserving model utility. The authors propose LEAF-OPD, a two-stage framework. First, they introduce the Supervision Signal (SS) metric to assess the strength and persistence of teacher supervision, and apply lightweight SFT to strengthen teachers with weak supervision signals. Second, a frozen teacher provides token-level supervision along student-generated trajectories. Experiments on TOFU, RWKU, and MUSE-News demonstrate promising performance.

Critical strengths
  1. The paper identifies an important observation that a teacher may generate correct refusals while providing weak supervision beyond the initial tokens of student-generated factual trajectories, and demonstrates that lightweight SFT can substantially improve the strength and persistence of such supervision.

  2. The proposed method achieves strong empirical performance, particularly on TOFU and RWKU, while the ablation studies provide useful evidence supporting the key design choices, including teacher strengthening, KL divergence direction, and retain-side supervision.

Critical weaknesses
  1. Limited novelty. The contribution beyond existing distillation-based unlearning methods, such as DUET [1], is unclear. The paper should discuss the methodological differences and include direct performance comparisons.

  2. Unclear contributions of teacher SFT and student initialization. Section 4.1 and Appendix C.2 state that the model obtained after Stage 1 SFT serves as both the frozen teacher and the student’s initial checkpoint for Stage 2. Table 4 compares good and bad teachers, but does not specify whether the student initialization is held fixed. If changing the teacher also changes the student’s starting parameters, the observed improvement cannot be attributed solely to better teacher supervision. Please clarify the student initialization used in these experiments. If it changes between the good- and bad-teacher settings, an additional comparison should use the same student initialization while varying only the teacher. This would help isolate the effect of teacher SFT on the supervision it provides. Please also report the forgetting and retention performance of the Stage 1 checkpoint without privileged context and before Stage 2 distillation (SFT-only). Comparing this checkpoint with the final student would show what changes have already occurred during SFT and what additional improvement comes from distillation.

  3. Limited evaluation across forget-set sizes: the experiments on TOFU are restricted to the 1% forgetting setting, which involves only 2 out of 200 fictional authors. Since TOFU also provides 5% and 10% forgetting settings, evaluating the method under these larger forget sets would help establish whether its effectiveness and forgetting–retention trade-off generalize to larger unlearning scales.

[1] DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teacher. ICLR 2026

Questions for authors
  1. If the teacher undergoes SFT in Stage 1, what motivates initializing the student in Stage 2 from the post-SFT checkpoint rather than the original model? Additionally, why is the post-SFT teacher also used for retain-side distillation instead of the original model? Would using the original model as the retain-side teacher provide a stronger constraint for preserving model utility?

  2. Could the authors clarify the exact construction of the Stage-1 SFT training data for TOFU and RWKU? In particular, are the GPT-5.6-Luna-generated wrong answer examples in TOFU used as factual prefixes followed by refusal responses in the SFT training sequences? Similarly, how are the wrong answer fields in the locally generated RWKU synthetic records incorporated into the SFT training sequences?

4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer W7M4

We thank you for your time, effort, and valuable comments; below are our responses to your comments.

Weakness 1. Clarify the methodological novelty over existing distillation-based unlearning methods such as DUET and provide direct performance comparisons.

We acknowledge that DUET and LEAF-OPD share the general idea of distilling unlearning behavior from a context-conditioned teacher. LEAF-OPD differs in its focus on the effectiveness of teacher supervision along student-generated trajectories: a teacher that can independently generate refusals may not provide sustained corrective supervision after the student has already generated part of the target answer. To address this issue, we use SS to assess both the overall strength of supervision and its persistence in the latter half of the response, apply lightweight SFT when necessary, and perform token-level distillation along the current student’s on-policy trajectories. In contrast, DUET’s main method distills Top-K logits at the first decoding position, while its multi-step extension uses teacher-generated trajectories. Thus, DUET is off-policy, whereas LEAF-OPD is on-policy.


Weakness 2. Clarify whether student initialization is held fixed in the teacher-quality ablation, isolate teacher quality using a common student initialization, and report SFT-only forgetting and retention performance without privileged context.

Thank you for identifying this important confounding factor. In the original Table 4 experiments, the good-teacher setting uses the Stage-1 SFT checkpoint as both the teacher and the student’s initialization, whereas the bad-teacher setting uses the original model for both. We therefore acknowledge that this comparison reflects the combined effects of teacher SFT and changes in student initialization, rather than isolating the contribution of teacher supervision quality.

To distinguish these two factors, we will add controlled experiments that hold student initialization fixed while varying only the teacher, keeping all other training configurations unchanged.

Method / SettingFQ ↑MU ↑R-RL ↑F-RL ↓
LEAF-OPD (self distillation)0.01430.68920.93820.0027
LEAF-OPD (different models)0.09710.66820.94520.0027

The results show that LEAF-OPD is effective with a good teacher under both student initialization settings. We would also like to clarify that LEAF-OPD was developed within the on-policy self-distillation (OPSD) framework, in which the teacher and student policies start from the same model checkpoint. In the configuration you suggested, reported as “LEAF-OPD (different models),” the teacher and student start from different checkpoints. This configuration therefore falls under on-policy distillation (OPD) in the strict sense

We will also report FQ, MU, F-RL, and R-RL for the Stage-1 SFT checkpoint without privileged context or Stage-2 distillation, and compare these results with the final model to quantify the additional changes brought by Stage-2 distillation. Please refer to Response to Common Concerns Raised by Reviewers - 1.The tiny SFT on stage1 - b.The effect of tiny SFT on performance. We will also add this analysis to Ablation 6 in Section 6 (Ablation Study).


Weakness 3. Evaluate TOFU 5% and 10% forget sets to assess whether effectiveness and the forgetting–retention trade-off generalize to larger unlearning scales.

We will report the results for the TOFU 5% and 10% settings below and include them in the revised paper in Appendix D.

Methodforget05 FQ ↑MU ↑R-RL ↑F-RL ↓forget10 FQ ↑MU ↑R-RL ↑F-RL ↓
GA6.57e-1207.35e-50.0037342.52e-14000
CL1.12e-280009.63e-46000
GradDiff1.21e-10000.0008521.58e-1700.00970.0096
KL3.08e-1207.35e-50.0011172.07e-1300.00970.0096
IDK1.21e-100.5283220.6869780.0854722.57e-180.55560.77100.0639
DPO2.61e-70.0307120.0111690.0119638.57e-110.02590.01140.0104
NPO0.3281150.3394270.3220520.2725160.00700.18830.16830.1359
LLMU2.57e-14000.0011171.43e-700.01740.0059
FLAT-Pearson4.42e-2700.00011405.30e-45000
LEAF-OPD9.59e-100.67260.94800.00718.68e-180.67730.93630.0069

Question 1. Explain why the post-SFT checkpoint is used for both student initialization and retain-side teacher supervision instead of the original model, and whether the original model would better preserve utility as the retain-side teacher.

Please refer to our responses in the Weakness section.


Question 2. Clarify how Stage-1 SFT data are constructed for TOFU and RWKU, particularly how generated wrong answers are incorporated and whether they serve as prefixes followed by refusal responses.

Thank you for pointing out this ambiguity. For Stage-1 SFT, we construct 20 synthetic records for TOFU and 240 fictional-person records for RWKU, the latter using fixed-seed local sampling from GPT-generated word lists and template-based questions and answers. Here, wrong_answer denotes a response that discloses target information and violates the unlearning requirement. We use its prefix as already-generated assistant content and append a target refusal, computing the loss only on the refusal. Conditioned on the privileged context, question, and prefix, the teacher thus learns to switch to refusal after partial disclosure, supporting sustained corrective supervision along student-generated trajectories.


We look forward to your response and would be happy to discuss any aspect of the paper with you.

The authors have posted their responses to your comments. Could you please take a look and let us know whether your concerns have been addressed? We encourage active discussion between reviewers and authors during this period.

Dear Reviewer W7M4,

We appreciate your careful review and constructive feedback. We are writing to check whether our responses have sufficiently clarified the points you raised. Should any questions remain unanswered or require further explanation, we would welcome the opportunity to discuss them with you. During the remaining rebuttal period, we will be ready to provide further clarification and additional experimental results to address your concerns.

Thank you once again for the time and effort you have devoted to reviewing our work!

Reviewer_7incOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an LLM unleanring framework that uses on-policy distillation (OPD). Aligning the model to a static refusal response is insufficient to teach deep refusal as the training signal is concentrated in the beginning of the refusal response. The paper utilizes OPD to effectively mitigate this. The teacher in OPD scores the student's own generation at every position, which offers training signal adapted to the student's behavior. As training progresses, the teacher provides training signal that can address partial unlearning. The experimental result shows that LEAF-OPD achieves beest forget quality and model utility on TOFU 1% dataset with Llama 2-7B and Qwen3-8B models.Similarly, LEAF-OPD achieves best forget quality on RWKU and MUSE-NEWS dataset.

Critical strengths
  1. The motivation is well defined. Introducing OPD to unlearning seems to be a favorable and efficient design choice over reinforcement learning.
  2. The paper is well written and was easy to follow.
  3. The proposed methods' retain loss is also supervised by the teacher. This design is clever as it re-uses the teacher model and increases the model utility drastically compare to the baselines (on TOFU dataset).
Critical weaknesses
  1. Teacher needs a small supervised fine-tuning (SFT) to maximize effectiveness of the proposed method. This implies that the level of teacher training affects the performance. While the paper describes the size of dataset and epochs for each teacher, it does not describe (1) how to train the teacher and (2) how to obtain SFT data. If a system operator has a dataset to unlearn from a model, then how should he (1) collect SFT data for the teacher and (2) determine whether teacher is sufficiently learned? The paper needs to have a dedicated section for this, since it is one of the most critical part of the proposed method.

  2. Table 1: Result of FLAT (Pearson) The first two rows of the table (Original LLM and Retrained LLM) for Llaama2-7B has exactly same numbers with FLAT paper (Wang et al, 2024). That shows that the experiment used the identical original and re-trained model that were used from the previous paper. However, FLAT (Pearson)'s result on the table is different from what was reported on FLAT paper. From FLAT paper, FQ, MU, F-RL and R-RL are (0.0541 0.6374 0.4392 0.8857), while the metric reported in the table is (0.0030 0.6304 0.4825 0.9284). The authors need to report why FQ from the table is significantly different from what was reported in the original FLAT paper.

  3. Table 1: Abnormally high model utility. Comparing the Original LLM and LEAF-OPD, MU of LEAF-OPD is significantly higher than that of original LLM (0.6892 vs 0.6239). However, R-RL of LEAF-ODE is lower than that of original LLM (0.9382 vs 0.9818). How could LEAF-OPD achieve better MU when its R-RL is lower? The authors need to provide clarification on the result.

  4. The paper does not include any unlearning robustness analysis, such as relearning attack.

  5. The experiment does not include large model (13B or bigger). This is critical because cost of teacher SFT could explode despite of small number of iterations. Hence it is necessary to analyze the cost and feasibility of training larger models. Subsequently, the experiment should demonstrate the scalability of the proposed method on the larger models.

  6. The paper does not include any efficiency analysis. The proposed method could be slower compare to the baselines since it requires (1) teacher SFT and (2) forward pass on both teachers and students for each iteration. While being inefficient could be acceptable, the paper should include the efficiency analysis that shows the runtime of the proposed method and baselines and the analysis should offer a justification on inefficiency (if the proposed method is less efficient.)

Questions for authors
  1. Please see the weaknesses and address the concerns.
  2. Why only TOFU 1% was reported? Could you report 5% and 10% dataset?

I am willing to update the score depending on the authors' response. FQ of FLAT from Table 1 is particularly concerning to me. If rebuttal could address it, I will increase my score.

6 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer 7inc

We thank you for your time, effort, and valuable comments; below are our responses to your comments.

1. Clarify how the teacher is fine-tuned, how its SFT data are constructed, and how sufficient teacher training is determined.

For Stage 1 of LEAF-OPD in each experiment, where a bad teacher is converted into a good teacher through lightweight SFT, Appendix A provides the detailed prompts used to generate the SFT data and specifies the generation models. To assess whether the teacher has been sufficiently trained, we use the Supervision Signal (SS) metric described in Sections 4.1 and 4.2. These sections explain how SS helps distinguish good teachers from bad teachers and how the SFT data are obtained. We have highlighted these explanations in red.


2. Explain why the FLAT (Pearson) results, particularly FQ, differ from those reported in the original FLAT paper despite matching Original and Retrained LLM results.

In the TOFU [1] benchmark, the authors publicly released the original and retrained Llama2-7B models on Hugging Face, so we report the same values in the first two rows. All baseline results in our table, however, were obtained from our own experiments under the same fixed hyperparameter settings (see Appendix C). We conducted training and evaluation with different random seeds, and the results are shown below:

Seed / MethodForget Quality ↑Model Utility ↑Retain ROUGE ↑Forget ROUGE ↓
400.00680.62410.91800.4384
1230.00300.62870.89040.4394
2340.00680.62550.89350.4227
3450.01430.62120.90700.4183
420.00300.63040.92840.4825
Mean ± Sample SD0.00678 ± 0.004610.62598 ± 0.003660.90746 ± 0.016090.44026 ± 0.02539
LEAF-OPD (Ours)0.01430.68920.93820.0027

These results indicate that we could not exactly reproduce the results reported in the original FLAT [2] paper, possibly due to differences in configurations and hardware. Nevertheless, while FQ differs from the originally reported value, FLAT achieves a higher retain ROUGE-L (R-RL) in our experiments.


3. Explain why LEAF-OPD achieves higher model utility than the Original LLM despite having lower retain ROUGE-L.

Thank you for the question. We would like to clarify that R-RL and MU differ in scope and in the dimensions they measure. As described in Section 5.2, R-RL measures only ROUGE-L on the retain set, whereas MU combines Probability, Truth Ratio, and ROUGE-L across the retain set, real-author questions, and world-fact questions. Thus, R-RL is only one component of the overall utility evaluation, and a decrease in R-RL does not necessarily imply a decrease in MU, as improvements in other components can offset it. Therefore, the lower R-RL and higher MU reported in Table 1 are not contradictory.


4. Evaluate unlearning robustness against attacks such as relearning.

Thank you for pointing out this limitation. This paper primarily evaluates LEAF-OPD’s forgetting effectiveness and utility preservation on standard unlearning benchmarks; we have not yet evaluated the unlearned model’s robustness to relearning attacks. Upon acceptance, we will clarify the scope of the current results and identify a systematic evaluation of relearning robustness under varying amounts of attack data and training budgets as future work.


5. Demonstrate scalability to models with at least 13B parameters and analyze the cost and feasibility of teacher SFT at that scale.

Thank you for pointing this out. We did not provide a detailed account of the Stage-1 lightweight SFT training time, but Table 6 reports the amount of data used in each stage for each experiment. Stage 1 uses substantially less data than Stage 2; on TOFU, it uses only 0.5% as much data. Across all experiments and models evaluated, lightweight SFT takes no more than 10 minutes under the configurations described in Appendix C. We therefore expect that scaling from 7B to 13B would not incur a substantial additional SFT cost.


6. Compare runtime with baselines, including teacher SFT and teacher/student forward-pass costs, and justify any additional overhead.

Compared with fine-tuning-based methods such as GA[3] and GD, LEAF-OPD takes longer to run. We report this comparison below and will also include the table in the paper (ablation 5, section 6), highlighted in red. However, LEAF-OPD runs faster than RL-based methods such as RULE[4], because these methods typically generate nn samples per prompt, whereas LEAF-OPD requires only one.

Runtime comparison on Qwen3-8B.

MethodRuntime (min)MethodRuntime (min)
GA5LLMU7
GD6NPO8
KL9FLAT10
PO9LEAF-OPD20
DPO8

7. Report results on the TOFU 5% and 10% forget sets in addition to the 1% setting.

We will report the results for the TOFU 5% and 10% settings below and include them in the revised paper in Appendix D.

Methodforget05 FQ ↑MU ↑R-RL ↑F-RL ↓forget10 FQ ↑MU ↑R-RL ↑F-RL ↓
GA6.57e-1207.35e-50.0037342.52e-14000
CL1.12e-280009.63e-46000
GradDiff1.21e-10000.0008521.58e-1700.00970.0096
KL3.08e-1207.35e-50.0011172.07e-1300.00970.0096
IDK1.21e-100.5283220.6869780.0854722.57e-180.55560.77100.0639
DPO2.61e-70.0307120.0111690.0119638.57e-110.02590.01140.0104
NPO0.3281150.3394270.3220520.2725160.00700.18830.16830.1359
LLMU2.57e-14000.0011171.43e-700.01740.0059
FLAT-Pearson4.42e-2700.00011405.30e-45000
LEAF-OPD9.59e-100.67260.94800.00718.68e-180.67730.93630.0069

[1] LLM Unlearning via Loss Adjustment with Only Forget Data, ICLR2025

[2] TOFU: A Task of Fictitious Unlearning for LLMs, COLM2024

[3] Large Language Model Unlearning, nips2024

[4] RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimality, nips2025

The authors have posted their responses to your comments. Could you please take a look and let us know whether your concerns have been addressed? We encourage active discussion between reviewers and authors during this period.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to Common Concerns Raised by Reviewers

We thank all reviewers and the AC for their hard work and valuable feedback! Below, we provide a consolidated response to common concerns raised by multiple reviewers:

1. The tiny SFT on Stage 1

a. Details of tiny SFT

In Appendix A, we describe how the data used for Tiny SFT are generated and report the SFT dataset size for each benchmark. As shown in Table 7, Stage 1 SFT uses substantially less data than Stage 2 OPSD training. Furthermore, to eliminate the influence of SFT on the final experimental results and use it solely as a tool to convert a bad teacher into a good teacher, we use generated data that are as unrelated as possible to the forget and retain data.


b.The effect of tiny SFT on performance

To assess the effect of SFT alone on performance, we directly evaluate the Llama2-7B model after Stage 1, i.e., the good teacher, before Stage 2 distillation. The results are shown below:

ModelFQ ↑MU ↑R-RL ↑F-RL ↓
Good Teacher (SFT only)0.00130.67150.94710.9323
LEAF-OPD (Ours)0.01430.68920.93820.0027

We observe that the good teacher obtained after SFT performs poorly across all metrics. We have further clarified this point in Ablation 6 of Section 6 (Ablation Study), with the added discussion highlighted in blue.

The authors have posted their rebuttal, and the discussion period will end soon. Could you please read it and reply at your earliest convenience, stating whether your concerns have been addressed and whether your rating changes? Thank you so much for your active participation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.