- Rating
- 3(+2)
- Confidence
- 3(+1)
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper views LLM unlearning from a continual learning-based perspective, and cast LLM unlearning as the inverse analogue of on-policy, reverse-KL fine-tuning. Under this formulation, it derives a novel, closed-form unlearning objective using negative-order Rényi divergence without a retain regularizer. This paper shows that this method achieves competitive forgetting on TOFU MUSE across different model sizes.
Critical strengths
- Simple objective. MARU needs no retain training data so the training objective is simpler and makes it easy to unlearn.
- Experimental coverage. This paper covers two benchmarks and different sizes of models making the results convincing.
Critical weaknesses
-
Utility degradation. In Table 1, 1B/Forget-10%, utility falls from 0.708 to 0.533, it is hard to judge MARU is better than the baselines.
-
Limited novelty. Divergence-based training objectives are well established, including KL-based retain losses in unlearning. Using negative-order Rényi divergence for the forget loss alone appears incremental without a clearer distinction from existing objectives. In particular, both MARU and NPO use model-to-reference likelihood ratios.
-
Missing baselines. The author mentions there are on-policy unlearning methods, but they do not include them in the experiments, such as PURGE from Reinforcement Unlearning via Group Relative Policy Optimization, and RULE from Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding
Questions for authors
- Why does MARU incur substantial utility loss despite its preservation claim (Table 1, 1B/Forget-10%)? Is the degradation caused by the lack of retain regularizer?
- How does MARU differ mathematically from NPO, both MARU and NPO use model-to-reference likelihood ratios? Can the author clarify the novelty compared with existing works?
- How does MARU compare with PURGE and RULE?
7 comments
Response to Reviewer t7aK (2/2)
W2 / Q2 (Novelty relative to NPO and KL-based losses): Both MARU and NPO use the ratio , but they weight it in opposite directions. Both gradients are weighted gradient ascent on the forget set. For NPO, ; for MARU at , with .
NPO's weight shrinks toward zero as an example is forgotten, which is what bounds NPO's progress. MARU's weight grows on already-suppressed examples and is normalized across the batch, so it keeps pushing forgotten examples further down rather than spreading effort evenly. This drives MARU's higher forgetting at Forget 10% and its peak-then-decline trajectory (Fig. 4), and it is why MARU needs early stopping. The KL-based losses the reviewer mentions (e.g., the GD retain term, Eq. 22) are attractive objectives applied to retain data. MARU applies a sign-flipped Rényi functional to the forget data, and it decomposes exactly (see our response to JFEZ, W1) into a likelihood-ratio term plus a genuine Rényi divergence on the forget set. To isolate the forget objective, we compare MARU against NPO and WGA with their retain losses removed under matched compute: MARU leads on both aggregates at Forget 10% but not at Forget 1% or 5% (see our response to JFEZ, W7).
W3 / Q3 (Missing method baselines, PURGE): We agree these belong in the comparison. One clarification: the RULE we cite is Zhang et al. (NeurIPS 2025), "RULE: Reinforcement Unlearning Achieves Forget-Retain Pareto Optimality"; we believe the reviewer intended this paper rather than the Leak@k paper. We have run PURGE, as a representative of the reinforcement-unlearning family, on TOFU at 1B for all three forget splits under the same protocol (3 seeds, table below). We implemented PURGE ourselves within OpenUnlearning, based on the authors' code. To keep the comparison fair, PURGE was trained with the same settings as all other methods: the same learning rate (), number of epochs (10) and checkpoint-selection protocol. To give it a stronger learning signal, we used an exponential reward and 8 sampled answers per question; the plain binary reward with 4 answers did even less (Mem 0.06 → 0.09 on Forget 1%). In this matched setting, PURGE preserves utility at the target level (0.70–0.72) but forgets little: Mem rises only to 0.16–0.26 and plateaus by about epoch 6, and Priv stays near 0.01, close to the original model. It therefore has the lowest HM(Mem, Util) of all methods at every split, and MARU outperforms it by 0.17 to 0.20 on HM(Mem, Util) and 0.28 to 0.49 on HM(Mem, Priv, Util) (paired t-tests, all ). We do not claim this is PURGE's best achievable performance: we report this as a matched-budget comparison.
PURGE vs. MARU, Llama-3.2-1B, TOFU, mean ± std over 3 seeds at the selected checkpoint. Ep. is the selected epoch, averaged over seeds. Higher is better; best per split and column in bold. The other baselines are in our response to JFEZ, W5.
| Split | Method | Mem | Priv | Util | HM(Mem, Util) | HM(Mem, Priv, Util) |
|---|---|---|---|---|---|---|
| F1 | PURGE | 0.158 ± 0.025 | 0.009 ± 0.002 | 0.724 ± 0.006 | 0.259 ± 0.033 | 0.026 ± 0.006 |
| F1 | MARU | 0.328 ± 0.004 | 0.297 ± 0.013 | 0.726 ± 0.002 | 0.452 ± 0.003 | 0.385 ± 0.008 |
| F5 | PURGE | 0.188 ± 0.017 | 0.007 ± 0.002 | 0.724 ± 0.004 | 0.298 ± 0.022 | 0.020 ± 0.006 |
| F5 | MARU | 0.397 ± 0.036 | 0.169 ± 0.051 | 0.659 ± 0.018 | 0.494 ± 0.024 | 0.297 ± 0.061 |
| F10 | PURGE | 0.259 ± 0.007 | 0.008 ± 0.001 | 0.699 ± 0.002 | 0.378 ± 0.007 | 0.023 ± 0.002 |
| F10 | MARU | 0.556 ± 0.017 | 0.459 ± 0.048 | 0.538 ± 0.019 | 0.547 ± 0.002 | 0.513 ± 0.018 |
Thank you for the detailed responses and additional experiments. My original concerns have largely been addressed, and I am open to raising my score. I would appreciate clarification on the following two remaining points; satisfactory answers would strengthen my case for doing so.
-
Removing the retain losses is a useful ablation. However, using identical learning rates and comparing fixed epochs does not establish a fair comparison of the forgetting–utility trade-off. At Forget10, GA/NPO/WGA retain utility of 0.607–0.624 at their selected checkpoints, versus 0.538 for MARU, but achieve weaker forgetting; meanwhile, MARU’s utility also drops from 0.538 at epoch 5 to 0.133 at epoch 10. These results do not distinguish a better trade-off from slower optimization or later utility collapse. Could you tune the learning rate and relevant method-specific parameters under comparable search budgets, and compare forgetting at matched retained utility? Reporting the resulting forgetting–utility frontiers would directly address this concern.
-
Thank you for clarifying the distinction from NPO. You explain that MARU’s weight increases on already-suppressed examples, so it “keeps pushing forgotten examples further down rather than spreading effort evenly.” This clarifies the mechanism, but why is allocating more weight to already-suppressed examples beneficial, rather than focusing on examples that remain memorized? Since a single sufficiently suppressed example can dominate Eq. 12, stronger suppression need not imply broader forgetting. Could you report the distribution of sample weights and per-example forgetting, particularly residual leakage among the least-forgotten examples, and compare against uniform weighting or NPO at matched retained utility? This would help establish whether the proposed weighting improves forgetting coverage or primarily over-suppresses a small subset.
Response to Reviewer t7aK (1/2)
We thank the reviewer for the points raised. We address the three concerns below with new analysis and experiments.
W1 / Q1 (Utility drop at 1B, Forget 10%): The reviewer is right that utility falls from 0.708 to 0.533 (0.538 ± 0.019 over 3 seeds, the lowest of all methods at this split; see our response to JFEZ, W5), and we now say plainly that MARU's lead at this split is a better forgetting–utility trade-off, not utility preservation. The drop is partly a consequence of removing the retain regularizer: all baselines except GA use an NLL retain loss (Table 6). Three new results put it in context:
Retain-free comparison, Llama-3.2-1B, TOFU, mean ± std over 3 seeds at the selected checkpoint (best HM(Mem, Util) per seed). Ep. is the selected epoch, averaged over seeds. Higher is better; best per split and column in bold.
Utility of the retain-free methods at fixed epochs, with no checkpoint selection (mean ± std over 3 seeds). 0.000 means utility collapsed to zero in every seed.