acceptodds
Under review as a conference paper at ICLR 2027

Mode-Avoiding Unlearning via Negative-Order Rényi Divergence

Abstract

Recent work on continual learning in Large Language Models (LLMs) shows that on-policy finetuning is implicitly biased toward the solution of minimal KL divergence from the base model, allowing reverse-KL-driven objectives to learn new knowledge while largely preserving prior capabilities. We argue that unlearning can be viewed as the inverse analogue of continual learning: rather than adding a mode while preserving the rest of the distribution, it must remove a specific forget mode while leaving the rest of the distribution untouched. Existing unlearning methods, however, typically rely on off-policy objectives that push the model toward an idealized unlearning target, leading to utility collapse. Since KL divergence is itself the limit of the R\'enyi divergence family, and every positive-order R\'enyi divergence only attracts a model toward a target distribution, we introduce MARU (Mode-Avoiding R\'enyi Unlearning), a method that uses the negative-order R\'enyi divergence as a natural mode-avoiding objective for unlearning. Anchored at the pre-unlearning policy, this yields an objective whose geometry implicitly preserves model utility without requiring an explicit retain set. Extensive experiments on the TOFU and MUSE benchmarks show that MARU achieves state-of-the-art performance on the hardest forget split (10%) at 1B scale, while delivering up to better resistance to membership inference attacks than comparable distillation-based methods, though these gains are weaker on MUSE. These results suggest that reframing unlearning through divergence geometry, rather than heuristic gradient objectives, offers a principled and effective path toward safe, controllable and utility-preserving unlearning in LLMs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine Unlearning, Continual Learning, Rényi Divergence, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_t7aKOpenReview ↗
Rating
3(+2)
Confidence
3(+1)
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper views LLM unlearning from a continual learning-based perspective, and cast LLM unlearning as the inverse analogue of on-policy, reverse-KL fine-tuning. Under this formulation, it derives a novel, closed-form unlearning objective using negative-order Rényi divergence without a retain regularizer. This paper shows that this method achieves competitive forgetting on TOFU MUSE across different model sizes.

Critical strengths
  1. Simple objective. MARU needs no retain training data so the training objective is simpler and makes it easy to unlearn.
  2. Experimental coverage. This paper covers two benchmarks and different sizes of models making the results convincing.
Critical weaknesses
  1. Utility degradation. In Table 1, 1B/Forget-10%, utility falls from 0.708 to 0.533, it is hard to judge MARU is better than the baselines.

  2. Limited novelty. Divergence-based training objectives are well established, including KL-based retain losses in unlearning. Using negative-order Rényi divergence for the forget loss alone appears incremental without a clearer distinction from existing objectives. In particular, both MARU and NPO use model-to-reference likelihood ratios.

  3. Missing baselines. The author mentions there are on-policy unlearning methods, but they do not include them in the experiments, such as PURGE from Reinforcement Unlearning via Group Relative Policy Optimization, and RULE from Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding

Questions for authors
  1. Why does MARU incur substantial utility loss despite its preservation claim (Table 1, 1B/Forget-10%)? Is the degradation caused by the lack of retain regularizer?
  2. How does MARU differ mathematically from NPO, both MARU and NPO use model-to-reference likelihood ratios? Can the author clarify the novelty compared with existing works?
  3. How does MARU compare with PURGE and RULE?
7 comments
AuthorsOpenReview ↗

Response to Reviewer t7aK (1/2)

We thank the reviewer for the points raised. We address the three concerns below with new analysis and experiments.

W1 / Q1 (Utility drop at 1B, Forget 10%): The reviewer is right that utility falls from 0.708 to 0.533 (0.538 ± 0.019 over 3 seeds, the lowest of all methods at this split; see our response to JFEZ, W5), and we now say plainly that MARU's lead at this split is a better forgetting–utility trade-off, not utility preservation. The drop is partly a consequence of removing the retain regularizer: all baselines except GA use an NLL retain loss (Table 6). Three new results put it in context:

  • Retain-free comparison: NPO and WGA with their retain losses removed, so every method is trained on the forget set only. At their selected checkpoints, retain-free GA, NPO and WGA keep more utility than MARU (0.607–0.624 vs. 0.538) but forget less (Mem 0.40–0.46 vs. 0.556), and they must be stopped at epoch 2–3. By epoch 5, all three have collapsed to zero utility, while MARU keeps 0.538 (tables below). The utility drop is therefore the cost of unlearning without retain data, and MARU withstands it longest among the retain-free objectives at this split.
  • Fixed-epoch results with no checkpoint selection: MARU leads at epoch 5 (0.547 ± 0.002 over 3 seeds) but falls to 0.222 ± 0.083 by epoch 10, so its lead depends on early stopping.
  • OpenUnlearning's three-way aggregate HM(Mem, Priv, Util), which the framework uses for ranking: MARU 0.513 ± 0.018 vs. 0.478 ± 0.013 for GD at 1B Forget 10% over 3 seeds, a lead that is not significant (p=0.10p = 0.10); at 3B and 7B, GD or GA lead. At Forget 1%, MARU keeps utility at 0.726, essentially that of the target model (0.718). We have also removed the claim that utility preservation follows implicitly from the objective's geometry.

Retain-free comparison, Llama-3.2-1B, TOFU, mean ± std over 3 seeds at the selected checkpoint (best HM(Mem, Util) per seed). Ep. is the selected epoch, averaged over seeds. Higher is better; best per split and column in bold.

SplitMethodEp.MemPrivUtilHM(Mem, Util)HM(Mem, Priv, Util)
F1GA90.427 ± 0.0020.537 ± 0.0060.709 ± 0.0010.533 ± 0.0010.534 ± 0.003
F1NPO100.424 ± 0.0020.313 ± 0.0020.707 ± 0.0040.530 ± 0.0020.430 ± 0.002
F1WGA90.507 ± 0.0020.298 ± 0.0130.706 ± 0.0020.590 ± 0.0010.445 ± 0.010
F1MARU100.328 ± 0.0040.297 ± 0.0130.726 ± 0.0020.452 ± 0.0030.385 ± 0.008
F5GA90.510 ± 0.0060.334 ± 0.0300.583 ± 0.0110.544 ± 0.0010.449 ± 0.018
F5NPO100.503 ± 0.0030.283 ± 0.0150.592 ± 0.0050.544 ± 0.0010.416 ± 0.011
F5WGA40.510 ± 0.0090.120 ± 0.0210.544 ± 0.0160.527 ± 0.0020.247 ± 0.030
F5MARU100.397 ± 0.0360.169 ± 0.0510.659 ± 0.0180.494 ± 0.0240.297 ± 0.061
F10GA30.457 ± 0.0050.164 ± 0.0090.609 ± 0.0020.522 ± 0.0030.302 ± 0.011
F10NPO30.454 ± 0.0020.149 ± 0.0050.607 ± 0.0000.520 ± 0.0010.284 ± 0.006
F10WGA20.404 ± 0.0010.034 ± 0.0010.624 ± 0.0020.490 ± 0.0010.090 ± 0.002
F10MARU50.556 ± 0.0170.459 ± 0.0480.538 ± 0.0190.547 ± 0.0020.513 ± 0.018

Utility of the retain-free methods at fixed epochs, with no checkpoint selection (mean ± std over 3 seeds). 0.000 means utility collapsed to zero in every seed.

MethodF1, ep. 5F1, ep. 10F5, ep. 5F5, ep. 10F10, ep. 5F10, ep. 10
GA0.715 ± 0.0050.706 ± 0.0020.652 ± 0.0060.577 ± 0.0150.0000.000
NPO0.712 ± 0.0010.707 ± 0.0040.652 ± 0.0050.591 ± 0.0040.0000.000
WGA0.705 ± 0.0070.698 ± 0.0040.041 ± 0.0190.0000.0000.000
MARU0.725 ± 0.0030.726 ± 0.0020.680 ± 0.0060.659 ± 0.0180.538 ± 0.0190.133 ± 0.058
AuthorsOpenReview ↗

Response to Reviewer t7aK (2/2)

W2 / Q2 (Novelty relative to NPO and KL-based losses): Both MARU and NPO use the ratio πθ/πref\pi_\theta / \pi_{\mathrm{ref}}, but they weight it in opposite directions. Both gradients are weighted gradient ascent on the forget set. For NPO, wi=2πθ(yi∣xi)β/(πθ(yi∣xi)β+πref(yi∣xi)β)w_i = 2 \pi_\theta(y_i \mid x_i)^{\beta} / (\pi_\theta(y_i \mid x_i)^{\beta} + \pi_{\mathrm{ref}}(y_i \mid x_i)^{\beta}); for MARU at α=−1\alpha = -1, wi=ri/∑jrjw_i = r_i / \sum_j r_j with ri=πref(yi∣xi)/πθ(yi∣xi)r_i = \pi_{\mathrm{ref}}(y_i \mid x_i) / \pi_\theta(y_i \mid x_i).

NPO's weight shrinks toward zero as an example is forgotten, which is what bounds NPO's progress. MARU's weight grows on already-suppressed examples and is normalized across the batch, so it keeps pushing forgotten examples further down rather than spreading effort evenly. This drives MARU's higher forgetting at Forget 10% and its peak-then-decline trajectory (Fig. 4), and it is why MARU needs early stopping. The KL-based losses the reviewer mentions (e.g., the GD retain term, Eq. 22) are attractive objectives applied to retain data. MARU applies a sign-flipped Rényi functional to the forget data, and it decomposes exactly (see our response to JFEZ, W1) into a likelihood-ratio term plus a genuine Rényi divergence on the forget set. To isolate the forget objective, we compare MARU against NPO and WGA with their retain losses removed under matched compute: MARU leads on both aggregates at Forget 10% but not at Forget 1% or 5% (see our response to JFEZ, W7).

W3 / Q3 (Missing method baselines, PURGE): We agree these belong in the comparison. One clarification: the RULE we cite is Zhang et al. (NeurIPS 2025), "RULE: Reinforcement Unlearning Achieves Forget-Retain Pareto Optimality"; we believe the reviewer intended this paper rather than the Leak@k paper. We have run PURGE, as a representative of the reinforcement-unlearning family, on TOFU at 1B for all three forget splits under the same protocol (3 seeds, table below). We implemented PURGE ourselves within OpenUnlearning, based on the authors' code. To keep the comparison fair, PURGE was trained with the same settings as all other methods: the same learning rate (10−510^{-5}), number of epochs (10) and checkpoint-selection protocol. To give it a stronger learning signal, we used an exponential reward and 8 sampled answers per question; the plain binary reward with 4 answers did even less (Mem 0.06 → 0.09 on Forget 1%). In this matched setting, PURGE preserves utility at the target level (0.70–0.72) but forgets little: Mem rises only to 0.16–0.26 and plateaus by about epoch 6, and Priv stays near 0.01, close to the original model. It therefore has the lowest HM(Mem, Util) of all methods at every split, and MARU outperforms it by 0.17 to 0.20 on HM(Mem, Util) and 0.28 to 0.49 on HM(Mem, Priv, Util) (paired t-tests, all p≤0.015p \le 0.015). We do not claim this is PURGE's best achievable performance: we report this as a matched-budget comparison.

PURGE vs. MARU, Llama-3.2-1B, TOFU, mean ± std over 3 seeds at the selected checkpoint. Ep. is the selected epoch, averaged over seeds. Higher is better; best per split and column in bold. The other baselines are in our response to JFEZ, W5.

SplitMethodMemPrivUtilHM(Mem, Util)HM(Mem, Priv, Util)
F1PURGE0.158 ± 0.0250.009 ± 0.0020.724 ± 0.0060.259 ± 0.0330.026 ± 0.006
F1MARU0.328 ± 0.0040.297 ± 0.0130.726 ± 0.0020.452 ± 0.0030.385 ± 0.008
F5PURGE0.188 ± 0.0170.007 ± 0.0020.724 ± 0.0040.298 ± 0.0220.020 ± 0.006
F5MARU0.397 ± 0.0360.169 ± 0.0510.659 ± 0.0180.494 ± 0.0240.297 ± 0.061
F10PURGE0.259 ± 0.0070.008 ± 0.0010.699 ± 0.0020.378 ± 0.0070.023 ± 0.002
F10MARU0.556 ± 0.0170.459 ± 0.0480.538 ± 0.0190.547 ± 0.0020.513 ± 0.018

Thank you for the detailed responses and additional experiments. My original concerns have largely been addressed, and I am open to raising my score. I would appreciate clarification on the following two remaining points; satisfactory answers would strengthen my case for doing so.

  1. Removing the retain losses is a useful ablation. However, using identical learning rates and comparing fixed epochs does not establish a fair comparison of the forgetting–utility trade-off. At Forget10, GA/NPO/WGA retain utility of 0.607–0.624 at their selected checkpoints, versus 0.538 for MARU, but achieve weaker forgetting; meanwhile, MARU’s utility also drops from 0.538 at epoch 5 to 0.133 at epoch 10. These results do not distinguish a better trade-off from slower optimization or later utility collapse. Could you tune the learning rate and relevant method-specific parameters under comparable search budgets, and compare forgetting at matched retained utility? Reporting the resulting forgetting–utility frontiers would directly address this concern.

  2. Thank you for clarifying the distinction from NPO. You explain that MARU’s weight increases on already-suppressed examples, so it “keeps pushing forgotten examples further down rather than spreading effort evenly.” This clarifies the mechanism, but why is allocating more weight to already-suppressed examples beneficial, rather than focusing on examples that remain memorized? Since a single sufficiently suppressed example can dominate Eq. 12, stronger suppression need not imply broader forgetting. Could you report the distribution of sample weights and per-example forgetting, particularly residual leakage among the least-forgotten examples, and compare against uniform weighting or NPO at matched retained utility? This would help establish whether the proposed weighting improves forgetting coverage or primarily over-suppresses a small subset.

Reviewer_VitjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Mode-Avoiding Rényi Unlearning (MARU), an LLM unlearning method motivated by minimizing negative-order Rényi divergence, equivalently maximizing a positive-order divergence with reversed arguments. Its practical objective uses reference-normalized likelihood ratios to adaptively weight updates on forget examples, without an explicit retain-data loss. The authors evaluate MARU on TOFU across three model scales and forget ratios, and on MUSE News and Books, with comparisons against gradient-, preference-, and distillation-based baselines. MARU achieves the highest reported forgetting–utility aggregate on TOFU Forget10 at the 1B scale and higher reported privacy scores than the selected distillation baselines, while its advantages do not consistently extend to larger models or MUSE. Additional experiments examine the divergence order, clipping, training trajectories, and runtime.

Critical strengths
  1. An interesting perspective on unlearning. I find the idea of viewing unlearning as the inverse of continual learning compelling. Exploring negative-order Rényi divergence as a way to turn distribution matching into repulsion is also an interesting development of this idea. Although the connection to utility preservation still needs stronger support, this perspective offers a useful motivation for a simple objective that does not require an explicit retain-data loss.

  2. Encouraging results supported by a reasonably broad evaluation. I appreciate that the authors evaluate MARU across multiple model scales and forget ratios, and include MUSE alongside TOFU. The improvement on TOFU Forget10 at the 1B scale is encouraging, particularly given the absence of an explicit retain loss. However, I am less convinced by its performance on Forget1 and Forget5, and the gains do not consistently carry over to larger models or MUSE. These results suggest that the approach has potential, although its practical advantages currently appear confined to a relatively narrow setting.

Critical weaknesses
  1. The theoretical results support a narrower claim than the paper suggests. Proposition 1 establishes that minimizing a positive-order divergence toward a fixed forget distribution encourages matching that distribution rather than moving away from it. This follows directly from the divergence’s nonnegativity and minimizer; it does not rule out unlearning by maximizing a positive-order divergence, nor establish that a negative-order objective is necessary. Moreover, the practical MARU objective involves two acknowledged approximations, but their effects on the theoretical interpretation remain insufficiently discussed. I would find the methodological justification more convincing with a clearer statement of the proposition’s scope and an analysis or discussion of what these approximations preserve.

  2. The central privacy claim lacks validation on MUSE. Privacy improvement is a major claimed contribution, yet the MUSE experiments report only memorization and retained-knowledge metrics, omitting the standard PrivLeak evaluation. Consequently, these results do not establish whether the reported privacy benefits extend beyond TOFU to real-world text. Reporting PrivLeak for the existing MUSE checkpoints and relevant baselines would directly address this gap without requiring a new training study.

Questions for authors
  1. Which properties of the ideal divergence objective survive the two approximations in Sec. 4.3? In particular, is there theoretical or empirical evidence that the resulting reference-normalized weighting preserves utility, beyond its interpretation as a repulsive objective? A discussion of the assumptions and limitations would help assess this connection.

  2. Can the authors report standard MUSE PrivLeak results for the existing MARU checkpoints and relevant baselines? If this evaluation is unavailable, what evidence supports extending the privacy claim beyond TOFU, and would the authors restrict its stated scope accordingly?

I think these questions concern the interpretation and validation of the current claims.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Vitj

We thank the reviewer for the positive assessment, and for finding the inverse-continual-learning view and the repulsive use of negative-order Rényi divergence novel and interesting. Both questions concern how far the current claims are supported, and we have narrowed them accordingly.

W1 / Q1 (Scope of Proposition 1 and what survives the approximations): We agree Proposition 1 supports a narrower claim than the text suggested. It shows only that minimizing a positive-order divergence toward πf\pi_f is attractive. It does not rule out unlearning by maximizing a positive-order divergence, and indeed by Proposition 2 MARU is exactly such an objective: minimizing D−1(πθ∥πf)D_{-1}(\pi_\theta \Vert \pi_f) is equivalent to maximizing D2(πf∥πθ)D_2(\pi_f \Vert \pi_\theta). We will update Section 4.1 to state this. The negative-order view is now presented as a natural way to obtain the repulsive objective within the Rényi family, not as a necessity.

On the two approximations in Section 4.3, we added an exact decomposition of Eq. 12 (details in our response to JFEZ, W1). On the finite space of forget pairs, Eq. 12 equals a mean-likelihood-ratio term plus a genuine Rényi divergence between a likelihood-ratio-tilted distribution and the uniform forget distribution, so Proposition 2 applies exactly there. What survives: the repulsive sign and the equivalence to maximizing an order-2 divergence. What does not: any guarantee about behavior outside DfD_f. Its gradient is a self-normalized reweighting of gradient ascent whose weights grow on already-suppressed examples, the opposite of NPO and WGA. Empirically, MARU without a retain loss is competitive with retain-based baselines at Forget 10%; compared with NPO and WGA with their retain losses removed, it leads at Forget 10% and is the only retain-free method that still has nonzero utility at epoch 5 there, but it does not lead at Forget 1% or 5% (see our response to JFEZ, W7). The final version will state these assumptions and limitations in an appendix, together with the full derivation.

W2 / Q2 (Privacy on MUSE): We agree this was a gap. We computed MUSE PrivLeak for all existing MARU and baseline checkpoints on News and Books. On News, MARU's PrivLeak is −99.3%, essentially the target model's −99.8% and far from the retrained model's −4.8%. On Books it is −59.1%, close to the target's −57.4%. No method reaches the [−5%, 5%] band on either corpus; the closest are GradDiff on News (−49.7%) and UNDIAL on Books (−48.9%). In other words, none of the evaluated methods, MARU included, achieves privacy on MUSE: every unlearned model remains clearly distinguishable from the retrained model as having seen the forget set. These values are at each method's selected checkpoint; Its later checkpoints overshoot to positive PrivLeak (+16% to +34% on News) as retain-set utility collapses, moving from under- to over-unlearning, and no evaluated checkpoint falls in the band. The TOFU privacy advantage therefore does not carry over to MUSE, as is also the case for the state-of-the-art baselines, and we will restrict the privacy claim to TOFU in the abstract, Section 5.2 and the conclusion.

MUSE PrivLeak (%) for Llama-2-7b, same checkpoints as Table 2. Success band is [−5, 5]; negative values indicate under-unlearning.

MethodNewsBooks
Target−99.8−57.4
Retrain−4.88.0
GA−98.5−59.2
GD−49.7−59.1
SimNPO−99.9−58.1
RMU−99.8−55.7
UNDIAL−99.7−48.9
DUET−99.8−57.8
MARU−99.3−59.1
Reviewer_VitjOpenReview ↗

Keep my rating

I maintain my Weak Accept recommendation.

The response addresses my concerns about the scope of Proposition 1 and the interpretation of the practical objective. I also appreciate the additional MUSE privacy evaluation and the authors’ willingness to narrow their claims.

Thank you.

Thank you for your thoughtful review and valuable feedback. We are glad we addressed your concerns.

Reviewer_JFEZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces MARU, a retain-free forgetting objective based on negative order Rényi divergence. The authors construct forgetting as the inverse process of continuous learning: instead of retaining existing behaviors while learning new target patterns, they remove specified forgetting patterns while minimizing incidental changes. They derive a signed negative order Rényi objective and instantiate this objective using the likelihood on the forgetting samples of the reference model.

MARU was evaluated on TOFU and MUSE using Llama-3.2-1B/3B and Llama-2-7B models. This method was compared with baseline methods based on gradients, preferences, representations, and distillation. The most significant result was achieved on the TOFU dataset with a 1B scale and a 10% forgetting ratio: MARU achieved the best overall score and achieved a higher calibration privacy score ratio on multiple distillation baselines. However, its advantage significantly weakened when the model size was larger, and it did not reach the current optimal level on MUSE.

Critical strengths
  1. The paper establishes a clear and potentially useful connection between continuous learning geometry and machine forgetting.
  2. The negative-order Rényi formula is mathematically interesting, and the paper explicitly discusses its signed properties and unboundedness.
  3. The experiments cover multiple model scales, two benchmarks, and a relatively wide set of baselines.
  4. The paper includes useful ablation experiments on Rényi order and clipping.
  5. The authors acknowledge that the transfer effect of MARU to MUSE is poor, and its advantages diminish as the model scale increases.
Critical weaknesses
  1. The realized loss is not the theoretical divergence analyzed in the main argument. The theory focuses on Dα(πθ∣∣πf)\mathrm{D}_\alpha (\pi_\theta \mid\mid \pi_f), but Equation 12 replaces the unknown forgetting distribution with the reference model likelihood and evaluates the results on an empirically observed forgetting completion set. This is a heuristic alternative rather than an estimator established with its relationship to the ideal divergence. There are no bounds or propositions linking the minimization of Equation 12 to pattern avoidance or utility preservation.
  2. The claim of "retain-free" utility preservation has not been proven. Equation 12 is trained only directly on the forgetting samples. The fact that its gradient is weighted by reference normalization does not mean that irrelevant behaviors will remain unchanged. The paper neither provides theoretical guarantees nor conducts a sufficiently adversarial assessment of irrelevant capabilities. The actual objective may be closely related to the dynamic weighted gradient ascent loss, but the paper does not directly make this comparison.
  3. The reported results involve a large number of test set checkpoint selections. For each method and each partition, the best checkpoint is selected using the aggregated memory...
  4. The privacy claim may be misleading. The "up to 31.5 times" improvement is a ratio relative to the mean of several privacy scores that are extremely small. Such ratios are unstable near zero and do not constitute a privacy guarantee. The paper should report the original attack AUC, confidence interval, attack direction, and statistical uncertainty. Privacy has not been evaluated on MUSE.
  5. The scope of empirical evidence is narrow and lacks uncertainty estimation. The paper does not clearly report multiple random seeds, variance, confidence intervals, or significance tests. MARU is not the best on larger models and does not lead on any MUSE metric. Therefore, the broad claim of "generally superior forgetting geometry" should be narrowed.
  6. The main objective function is unbounded and numerically fragile. The authors chose the unclipped loss, although acknowledging that a single sample may dominate the objective and push it to negative infinity. The paper does not report gradient norms, likelihood ratio distributions, overflow/underflow behavior, or sensitivity to stopping and random seeds.
  7. The novelty claim requires a stronger positioning. The paper should directly compare MARU with confidence-weighted gradient ascent, WGA, and other gradients that may have similar sample weighting behavior. Regarding the claim of "being the first to use negative-order Rényi objectives for training", a more systematic analysis of related work is needed.
Questions for authors
  1. What are the probability space and forgetting distribution applicable to Proposition 2? Since Equation 12 uses empirical observation completion and reference model likelihood, can the authors provide a formal derivation or bound that links the achieved loss to the claimed divergence objective?
  2. What is the difference between the gradient of MARU and that of WGA or other dynamic weighted gradient ascent methods? Please provide a direct comparison that matches computing power.
  3. Can the authors only select checkpoints on the independent validation split and then conduct an untouched test evaluation once to re-run the main results?
  4. How many random seeds were used for each method? Please report the confidence interval, the original MIA AUC, the calibration value, and the statistical test for the reported privacy improvement.
  5. What happens to the irrelevant ability under stronger detection, distribution shift, and longer training? Please report the gradient/loss stability and the frequency of numerical failure caused by unbounded objectives.
7 comments
AuthorsOpenReview ↗

Response to Reviewer JFEZ (1/5)

We thank the reviewer for a detailed and constructive review, and for recognizing the connection between continual-learning geometry and unlearning. The review identified real gaps, and we have revised the paper to close them.

W1 / Q1 (Relationship between Eq. 12 and the ideal divergence): We agree that, as submitted, Eq. 12 was presented as an instance of the ideal objective without a formal link. We now give an exact identity. Let the probability space be the finite set of forget pairs DfD_f, with QfQ_f uniform on it, and define the tilted distribution Pθ(i)∝πθ(yi∣xi)/πref(yi∣xi)P_\theta(i) \propto \pi_\theta(y_i \mid x_i) / \pi_{\mathrm{ref}}(y_i \mid x_i) with normalizer Z(θ)=EQf[πθ/πref]Z(\theta) = \mathbb{E}_{Q_f}[\pi_\theta / \pi_{\mathrm{ref}}]. Then Eq. 12 decomposes exactly as L(θ)=αα−1log⁡Z(θ)+Dα(Pθ∥Qf)\mathcal{L}(\theta) = \frac{\alpha}{\alpha-1} \log Z(\theta) + D_\alpha(P_\theta \Vert Q_f), where the first term is the mean likelihood ratio and the second is a genuine Rényi divergence, with Dα(Pθ∥Qf)=α1−αD1−α(Qf∥Pθ)D_\alpha(P_\theta \Vert Q_f) = \frac{\alpha}{1-\alpha} D_{1-\alpha}(Q_f \Vert P_\theta).

Both PθP_\theta and QfQ_f have full support on DfD_f, so Proposition 2 applies exactly on this space. At α=−1\alpha = -1 the first term is 12log⁡Z\tfrac12 \log Z, which pushes forget likelihoods down relative to the reference. The second term is −12D2(Qf∥Pθ)-\tfrac12 D_2(Q_f \Vert P_\theta), whose minimization makes the relative-likelihood profile across forget examples non-uniform. The only approximation remaining is πf:=πref\pi_f := \pi_{\mathrm{ref}}, which is a natural choice because the reference policy is the same policy that knows about the forget targets that we want to remove; we now state it as a modeling choice rather than an estimator. The full proof will be added as an appendix in the final version, and the revised Section 4.3 no longer describes Eq. 12 as a plug-in estimator of Dα(πθ∥πf)D_\alpha(\pi_\theta \Vert \pi_f).

W2 (Retain-free utility preservation and relation to weighted gradient ascent): We agree the claim was not proven, and we have removed the statement that utility preservation follows implicitly from the geometry. The reviewer is also right that MARU is closely related to weighted gradient ascent. With ρi(θ)=log⁡πθ(yi∣xi)−log⁡πref(yi∣xi)\rho_i(\theta) = \log \pi_\theta(y_i \mid x_i) - \log \pi_{\mathrm{ref}}(y_i \mid x_i), differentiating Eq. 12 gives ∇θL=αα−1∑iwi(θ)∇θlog⁡πθ(yi∣xi)\nabla_\theta \mathcal{L} = \frac{\alpha}{\alpha-1} \sum_i w_i(\theta) \nabla_\theta \log \pi_\theta(y_i \mid x_i), with weights wi(θ)=eαρi(θ)/∑jeαρj(θ)w_i(\theta) = e^{\alpha \rho_i(\theta)} / \sum_j e^{\alpha \rho_j(\theta)}.

At α=−1\alpha = -1, wi∝πref/πθw_i \propto \pi_{\mathrm{ref}} / \pi_\theta, so MARU is gradient ascent with self-normalized weights that grow on examples that are already suppressed. WGA (wi=πθαw_i = \pi_\theta^{\alpha} per token) and NPO (wi=2πθβ/(πθβ+πrefβ)w_i = 2\pi_\theta^{\beta} / (\pi_\theta^{\beta} + \pi_{\mathrm{ref}}^{\beta})) both shrink weights on suppressed examples. A second-order expansion around πθ=πref\pi_\theta = \pi_{\mathrm{ref}} also shows L(θ)≈αα−1ρˉ+α22(α−1)Var^(ρ)\mathcal{L}(\theta) \approx \frac{\alpha}{\alpha-1} \bar{\rho} + \frac{\alpha^{2}}{2(\alpha-1)} \widehat{\mathrm{Var}}(\rho).

So MARU is rescaled gradient ascent at initialization, which explains why all α\alpha values share the same early trajectory in Fig. 3. To test the retain-free claim directly, we removed the retain losses from NPO and WGA, so that GA, NPO, WGA and MARU are all trained on the forget set only (tables in W7). Without any retain data, MARU degrades more slowly than the other forget objectives: at Forget 10%, GA, NPO and WGA all reach zero utility by epoch 5, while MARU keeps 0.538. At Forget 1% and 5% it has no such advantage. We now present retain-freeness as a practical property, not a guarantee. On irrelevant capabilities, see Q5.

AuthorsOpenReview ↗

Response to Reviewer JFEZ (2/5)

W3 / Q3 (Checkpoint selection on the test evaluation): We followed the OpenUnlearning protocol (Dorna et al., App. F.2), which selects by HM(Mem, Util) on the TOFU evaluation and excludes privacy metrics from selection. The framework's authors argue this is acceptable for unlearning: unlike standard ML, the forget set is known in advance, so evaluating on it during tuning is "a permissible approach to ensure thorough forgetting before deployment" (Dorna et al., App. F.2). They leave privacy out of tuning because it requires an i.i.d. holdout set and an oracle retain model, and they acknowledge that whether selection metrics must be kept separate from leaderboard metrics is still an open question. We agree with the reviewer that this is selection on the evaluation set, and that it favors methods with a sharp optimum such as MARU (Fig. 4). We have therefore (i) re-reported all methods at fixed epochs 5 and 10 with no checkpoint selection, and (ii) added the framework's three-way aggregate HM(Mem, Priv, Util). Under (i), MARU leads at 1B Forget 10% at epoch 5 (HM(Mem, Util) 0.547 ± 0.002 over 3 seeds) but falls to 0.222 ± 0.083 by epoch 10, when RMU leads; its lead therefore depends on early stopping, which we now state explicitly. Under (ii), MARU leads at 1B Forget 10% (0.513 ± 0.018 vs. 0.478 ± 0.013 for GD, not significant at p=0.10p = 0.10; see W5), but not at Forget 1% or 5%, nor at 3B or 7B, where GD or GA lead. Because Priv was never part of the selection objective, the privacy result was not targeted by checkpoint selection.

W4 / Q4 (Privacy claim): We agree the ratio was misleading and have removed "31.5×" from the abstract and conclusion. The table below reports, for each of the four attacks (LOSS, Zlib, Min-k, Min-k++): the raw AUC, the retrained-model AUC used for calibration, the calibrated score. AUC is computed with forget examples as positives and holdout examples as negatives, so values above the retrained model's AUC indicate residual membership signal. At 1B Forget 10%, MARU has the lowest AUC of all methods on every attack, but remains well above the retrained model (e.g., LOSS 0.744 vs. 0.387), so membership is still detectable (table below). We also note in the text that GA, GD and NPO reach higher Priv than MARU at Forget 1% and 5% (3-seed table in W5). MUSE PrivLeak for all existing checkpoints is in the second table below. MARU shows no privacy advantage there (News −99.3%, Books −59.1%, vs. −99.8% and −57.4% for the target model), so we will restrict the privacy claim to TOFU. We now describe these results as empirical membership-inference resistance on these benchmarks, not a privacy guarantee.

Raw membership-inference AUCs, Llama-3.2-1B, TOFU Forget 10%, selected checkpoints, seed 0. Forget examples are positives and holdout examples negatives; lower is better, and the retrained model is the reference.

MethodLOSSZlibMin-kMin-k++Priv
Retrain0.3870.3090.3820.477–
GA0.9020.9040.9170.7830.171
GD0.8140.7700.8030.5970.373
NPO0.9200.9150.9340.8700.137
RMU0.9140.9150.9210.8520.150
UNDIAL0.9800.9560.9920.9990.006
WGA0.9830.9810.9850.9600.032
MARU0.7440.7140.7410.5360.481

MUSE PrivLeak (%) for Llama-2-7b, same checkpoints as Table 2. Success band is [−5, 5]; negative values indicate under-unlearning.

MethodNewsBooks
Target−99.8−57.4
Retrain−4.88.0
GA−98.5−59.2
GD−49.7−59.1
SimNPO−99.9−58.1
RMU−99.8−55.7
UNDIAL−99.7−48.9
DUET−99.8−57.8
MARU−99.3−59.1
AuthorsOpenReview ↗

Response to Reviewer JFEZ (3/5)

W5 (Seeds, uncertainty and scope): Each original run used one seed. So we ran MARU and the five strongest baselines (GA, GD, NPO, RMU, WGA) on all three TOFU splits with 3 seeds each (54 runs). Each seed's checkpoint is selected exactly as in the paper, and the new seed-0 runs reproduce the original numbers to within 0.005 HM. We compare methods with paired t-tests, pairing runs by seed; with 3 seeds these have only 2 degrees of freedom, so we treat them as indicative. Results (table below):

At Forget 10%, MARU leads on HM(Mem, Util), 0.547 ± 0.002, ahead of every baseline by 0.018 to 0.080 (all p≤0.015p \le 0.015). On the three-way HM(Mem, Priv, Util) it beats GA, NPO, RMU and WGA (p≤0.032p \le 0.032), but its lead over GD (0.513 vs. 0.478) is not significant (p=0.10p = 0.10). The lead comes from the strongest forgetting and privacy at this split (Mem 0.556, Priv 0.459) at the cost of the lowest utility (0.538). The fixed-epoch result also holds across seeds: at Forget 10%, MARU's HM(Mem, Util) is 0.547 ± 0.002 at epoch 5 but 0.222 ± 0.083 at epoch 10. We have replaced "generally superior forgetting geometry" with: "MARU is a retain-free objective that leads on TOFU Forget 10% at 1B scale, consistently across seeds, but trails retain-based baselines at smaller forget ratios, larger scales and on MUSE, and its advantage depends on early stopping." Llama-3.2-1B, TOFU, mean ± std over 3 seeds at the selected checkpoint. Higher is better in every column; best per split and column in bold.

SplitMethodMemPrivUtilHM(Mem, Util)HM(Mem, Priv, Util)
F1GA0.427 ± 0.0020.537 ± 0.0060.709 ± 0.0010.533 ± 0.0010.534 ± 0.003
F1GD0.426 ± 0.0150.433 ± 0.0250.655 ± 0.0090.516 ± 0.0080.485 ± 0.013
F1NPO0.423 ± 0.0010.313 ± 0.0070.704 ± 0.0040.528 ± 0.0010.430 ± 0.004
F1RMU0.437 ± 0.0030.148 ± 0.0120.680 ± 0.0090.532 ± 0.0010.285 ± 0.015
F1WGA0.395 ± 0.0070.057 ± 0.0080.714 ± 0.0050.508 ± 0.0050.140 ± 0.015
F1MARU0.328 ± 0.0040.297 ± 0.0130.726 ± 0.0020.452 ± 0.0030.385 ± 0.008
F5GA0.510 ± 0.0060.334 ± 0.0300.583 ± 0.0110.544 ± 0.0010.449 ± 0.018
F5GD0.464 ± 0.0010.359 ± 0.0180.589 ± 0.0070.519 ± 0.0020.452 ± 0.011
F5NPO0.482 ± 0.0010.230 ± 0.0070.624 ± 0.0040.544 ± 0.0010.374 ± 0.007
F5RMU0.408 ± 0.0040.073 ± 0.0060.636 ± 0.0110.497 ± 0.0020.170 ± 0.010
F5WGA0.351 ± 0.0040.041 ± 0.0060.713 ± 0.0030.471 ± 0.0030.105 ± 0.013
F5MARU0.397 ± 0.0360.169 ± 0.0510.659 ± 0.0180.494 ± 0.0240.297 ± 0.061
F10GA0.457 ± 0.0050.164 ± 0.0090.609 ± 0.0020.522 ± 0.0030.302 ± 0.011
F10GD0.477 ± 0.0070.408 ± 0.0220.578 ± 0.0020.522 ± 0.0040.478 ± 0.013
F10NPO0.472 ± 0.0540.179 ± 0.0860.581 ± 0.0670.516 ± 0.0020.306 ± 0.084
F10RMU0.443 ± 0.0030.161 ± 0.0130.656 ± 0.0030.529 ± 0.0020.301 ± 0.015
F10WGA0.347 ± 0.0020.035 ± 0.0020.711 ± 0.0010.466 ± 0.0020.091 ± 0.005
F10MARU0.556 ± 0.0170.459 ± 0.0480.538 ± 0.0190.547 ± 0.0020.513 ± 0.018

Comments on the paper

2 comments

Dear reviewers Thank you for your thoughtful review. The authors have submitted their rebuttal, and the discussion period is now open. Please take a moment to read the authors' response and engage with them and your fellow reviewers to address any remaining questions or clarify key points. Your active participation during this phase is vital to reaching a fair and well-informed final recommendation.

Best regards, area chair

AuthorsOpenReview ↗

Manuscript Revision

We thank all reviewers for the constructive discussion. We have uploaded a revised manuscript with all the changes marked in red. Below we summarize them and point to where each concern is addressed.

1. What the realized loss optimizes (JFEZ W1, W2, W6; Vitj Q1; t7aK Q2).

  • Section 4.3 no longer presents Eq. 12 as a plug-in estimator of the ideal divergence. πf:=πref\pi_f := \pi_{\mathrm{ref}} is now stated as a modeling choice, and the section points to an exact characterization of the loss.
  • New Appendix A.4 (Analysis of the Practical MARU Loss) contains: the exact decomposition of Eq. 12 into a mean likelihood-ratio term plus a genuine Rényi divergence on the forget set, with proof; the gradient as self-normalized reweighted gradient ascent, compared with the NPO and WGA weights; the second-order expansion around initialization; and the source of the unboundedness together with the numerical behavior.

2. Narrowed theoretical claims (Vitj W1, JFEZ W7).

  • Section 4.1 and the introduction now state that positive orders are attractive under minimization, that repulsion within the positive-order regime requires maximization, and that the negative order is a natural way to obtain a repulsive objective, not a necessity.
  • The novelty claim now reads "the first to use the repulsive (negative-order Rényi divergence) regime for unlearning" (Section 3.3).
  • The abstract now states MARU's performance more accurately.

3. Utility preservation (JFEZ W2, t7aK W1). We removed the claims that utility preservation follows implicitly from the objective's geometry (introduction and Section 4.3). MARU is now described as retain-free, and Section 4.3 states that it requires early stopping.

4. Privacy (JFEZ W4, Vitj W2).

  • The "31.5×" ratio has been removed from the abstract and from the contribution statement, together with the ratio-based paragraph in Appendix B.
  • New in Appendix B: raw AUCs for all four attacks with the retrained reference, and MUSE PrivLeak for all existing MUSE checkpoints.

5. Fair comparison and the weighting (t7aK follow-up). Two new analyses in Appendix B, both on TOFU Forget 10% at 1B with all methods trained without a retain loss:

  • Forgetting at matched utility under a matched tuning budget of nine configurations per method, with 3 seeds for the best configuration of each. MARU forgets more below utility ≈ 0.62 and has higher privacy at every utility ≤ 0.65; above 0.62, all methods except WGA tie on forgetting.
  • Weight concentration and per-example forgetting, with MARU-uniform and clipped MARU as controls. The weighting concentrates on one example per micro-batch and buys depth on a subset of examples rather than coverage; clipping reduces the residual tail but gives up the frontier gain.

We thank the reviewers again for their time and for valuable feedback that has made the paper more rigorous.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.