- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies whether collateral damage from LLM unlearning depends on the model having learned the forget documents. It compares two fine-tuned copies of a model: one trained on the forget documents and another trained on different documents. Both undergo the same unlearning procedure. The paper reports similar damage under RMU and, in some settings, NPO, while GradDiff and SimNPO show greater dependence on prior learning. It recommends including this control in unlearning evaluations.
Critical strengths
- The proposed control addresses a reasonable evaluation question. Applying unlearning to a model that never learned the target documents can reveal side effects that a standard before-and-after comparison cannot separate. Matching the fine-tuning recipe also helps account for the difference between pretrained and fine-tuned models.
- The RMU observation is tested across several models, datasets, and evaluation measures. The authors also examine alternative pairings, neutral fine-tuning data, and generated answers. These checks make the reported pattern more credible within the studied settings, even though they do not fully support the broader interpretation.
Critical weaknesses
- The novelty is limited. The basic finding that an unlearning update can damage a model even when the target documents were never learned is unsurprising. RMU pushes activations toward a random target without checking whether the input was memorized. More importantly, Section 2 already discusses closely related observations in Kim et al. and SALMUBench. The paired design offers a different measurement, but the paper does not establish a substantial advance in understanding why the effect occurs or when it should disappear. The three contributions largely amount to one control and two sets of observations obtained with it. Extensive repetition strengthens confidence in an observation, but does not by itself create a deeper contribution.
- The central comparison does not identify the quantity claimed in the paper’s framing. The difference between taught and untaught damage measures the net effect of prior fine-tuning on the outcome of unlearning. It does not separate damage caused by removing knowledge from changes in the model’s general susceptibility to an update. These effects could offset each other, so equal total damage does not establish that knowledge removal contributed little to the taught model’s damage. Appendix A explicitly acknowledges this limitation, but the abstract still states that most damage does not come from removing knowledge. The untaught model also retains pretrained knowledge about the forget topic; it lacks the newly taught false statements, not all related knowledge. The TOFU check helps with this distinction, but does not resolve the causal interpretation.
- The claimed separation between methods is highly dependent on the control design and metric. In Appendix H, GradDiff’s log-probability damage ratio is around one half in the split-topic design, but rises to 2.55 at 80 steps in the main design. At 160 and 320 steps, its main-design intervals include one. The same appendix states that multiple-choice accuracy does not separate the methods in the split-topic setting. On WMDP, GradDiff’s ratio is 0.29 in log-probability but 1.40 on the contrastive score, reversing the conclusion about which copy is harmed more. These are not minor numerical variations. They show that the result is not yet a stable property of the unlearning method. The paper needs to explain these reversals rather than mainly report them.
- Much of the evidence comes from settings with degraded generation, making the practical interpretation unclear. Appendix E reports that removing every run where either copy produces degenerate output leaves only 40 of 224 MMLU runs. In those remaining runs, damage is too small to estimate the accuracy ratio reliably. The nondegenerate results on other datasets provide some support, but do not establish the headline quantitative conclusion for useful unlearning settings. The cross-method comparison has a related problem: GradDiff causes roughly five to eight times RMU’s log-probability damage on MedMCQA and WMDP, while the damage-matched RMU–GradDiff test on MMLU is inconclusive. Matching damage after running the experiments also does not ensure comparable forgetting. Results at independently selected settings that achieve meaningful forgetting while preserving usable generation would be more informative than pooling a large sweep.
- The presentation is poor and unnecessarily repetitive. Section 5 repeatedly lists ratios, intervals, run counts, and additional experiments, with little explanation connecting them. Important qualifications are scattered across long appendices, including the failure to separate methods under accuracy and the uncertainty in the damage-matched comparison. The extensive seed-by-seed reporting adds further length without resolving these central questions. Repeated runs are useful for checking stability, but their results should be consolidated rather than repeatedly presented as new evidence. Overall, the manuscript reads more like an experiment log than a focused scientific argument. A much shorter presentation should distinguish the robust RMU observation, the setting-dependent findings, and the conclusions that remain unsupported.
Questions for authors
- What assumptions allow the taught–untaught damage difference to separate harm caused by knowledge removal from changes in general susceptibility to unlearning? How does the interpretation in the abstract follow despite the limitation acknowledged in Appendix A?
- Why does GradDiff’s result change substantially between the main and split-topic designs, and sometimes reverse across metrics? Which conclusion about the method survives these changes?
- Can the authors establish the main quantitative claims at independently selected operating points with meaningful forgetting and nondegenerate generation? Do the differences between methods persist at comparable forgetting levels within the same control design?
- Beyond quantifying an already documented failure mode, what new explanation or prediction does this control provide? Can the authors demonstrate a concrete case where it changes a conclusion that existing evaluations would otherwise get wrong?