- Rating
- 2(+1)
- Confidence
- 4(+1)
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies which parameters should be updated during LLM unlearning and whether this choice should change during training. It proposes Intervention Score for initial parameter selection and DIR-R for selectively comparing alternative continuations at a later checkpoint. Evaluation includes a controlled fact-injection experiment, Natural-TOFU, and LACUNA.
Critical strengths
- The paper raises an interesting and important question: the parameters that store certain knowledge may not be the best parameters to modify during unlearning. This distinction between knowledge localization and intervention effectiveness is meaningful. It challenges a common assumption in localized unlearning and could help explain why accurate localization does not always lead to better forgetting.
- The paper attempts to study parameter selection from multiple perspectives. The controlled experiment provides an interesting observation: despite a localization AUROC of 0.981, the identified storage locations do not consistently lead to better unlearning outcomes. The authors also consider different unlearning objectives and include a static version of their method to isolate the effect of dynamic selection. These are useful experimental directions, although the current evidence is not sufficiently convincing.
Critical weaknesses
-
The experimental comparisons are questionable. In Natural-TOFU, the reported 19/20 positive comparisons are not obtained from direct experiments. Instead, Eq. (22) combines results from different evaluation sets. This even changes the ordering against GRAIL under SimNPO. Such constructed comparisons cannot replace direct, paired experiments. Moreover, the LACUNA experiments select only three baselines per objective from a larger pool, without a sufficiently clear justification. Different experimental panels are also difficult to reconcile. For example, the NPO score of DIR-R is 6.194 in Table 14 but 1.328 in Table 15. The different settings and their relationship need to be much clearer.
-
The improvement from DIR-R is very limited. Appendix C.7 reports gains of only 0.0024 and 0.0082 under NPO and SimNPO, despite increasing runtime by 33% and 52%, respectively. It is hard to see the practical value of such small improvements. More importantly, five static baselines outperform DIR-R in the primary four-field GradDiff evaluation. The main text instead emphasizes a three-field analysis that excludes Birth City after observing the results. Although this is disclosed, it makes the overall experimental evidence less convincing.
-
The methodology has several important issues. Eq. (10) defines Intervention Score using a ratio of gradient-based effects, but its motivation is weak. The score may be sensitive to the scaling of different losses and to a near-zero denominator. No convincing analysis or ablation supports this particular formulation. Furthermore, DIR-R does not actually recompute parameter importance at later checkpoints. It only selects among candidate subsets constructed from the initial ranking. It is therefore unclear whether the improvement comes from state-dependent adaptation or simply from evaluating multiple candidates. In Eq. (15), candidate continuations are selected using the terminal score , which is also used for performance reporting. The paper does not clearly explain whether selection and final evaluation use independent data. This raises concerns about evaluation bias.
-
The model selection is outdated and insufficient. Natural-TOFU relies on Llama 3 and Gemma 2, both released in 2024. Given that Gemma 4 and Qwen3.5 are now available, experiments on more recent models would be much more meaningful.
-
The presentation quality is very poor. The entire paper contains no figures or analysis plots, not even a basic illustration of the proposed framework. Instead, the authors repeatedly list numerical results in the main text, often reporting values to six decimal places without meaningful interpretation. Many paragraphs simply repeat information already available in tables. This makes the paper unnecessarily long and difficult to follow. For a method built around dynamic parameter selection, the absence of any visualization of how parameter importance or selected subsets change during training is especially concerning. Overall, the manuscript reads more like a collection of experiment logs than a well-organized research paper.
Questions for authors
-
In Eq. (15), are the data used for candidate selection completely separate from those used for final evaluation? Can the authors clarify this protocol and report results on an independent test set?
-
Why are the Natural-TOFU comparisons constructed by combining results from different evaluation sets rather than directly evaluating all methods under identical settings? Can the authors provide direct comparisons?
-
How were the baselines selected for the LACUNA experiments? Why are only three baselines reported for each objective? Can the authors explain the differences between Tables 14 and 15 and provide complete comparisons under consistent settings?
-
What justifies the specific formulation of Intervention Score in Eq. (10)? How sensitive is it to loss scaling and the denominator? Can the authors also provide precise definitions of all diagnostic losses and evaluation metrics?
-
Can the authors demonstrate that DIR-R offers meaningful improvements over simpler alternatives, such as random support replacement or compute-matched candidate search? Does its advantage remain when accounting for the total number of modified parameters and additional computation?
-
Can the authors provide results on more recent model families, particularly Qwen3.5 or Gemma 4, and include figures illustrating the framework and the actual changes in parameter selection during training?
4 comments
A Gentle Follow-up on Our Rebuttal — Reviewer ScPL
Dear Reviewer ScPL,
Thank you again for your detailed review and for identifying several important concerns regarding the experimental evidence and presentation of our work.
As the author–reviewer discussion period approaches its conclusion, we would greatly appreciate any further feedback on our rebuttal and revised manuscript.
We have made substantial efforts to address your concerns by clarifying the constructed Natural-TOFU comparisons, reporting direct paired results, distinguishing initial intervention selection from checkpoint-dependent revision, and providing more transparent computational-cost accounting.
We have also added results on a newer Qwen model, evaluated branch selection on held-out queries, compared DIR-R with simpler search alternatives, and revised the presentation with additional figures and clearer explanations.
Importantly, we have explicitly reported the negative and near-zero findings, including cases where simpler alternatives outperform DIR-R, to ensure that our conclusions remain supported by the available evidence.
We sincerely hope these revisions clarify the contributions and limitations of our method. We would be grateful to know whether they address any of your principal concerns, or whether important issues remain unresolved.
Thank you very much for your time and consideration.
Best regards,
Authors of Paper 29979
Thank you for the response. I appreciate the additional experiments and clearer LACUNA comparisons, which address some of my concerns. However, the 19/20 result on Natural-TOFU still combines estimates from different query sets. DIR-R’s gains over fixed support remain small relative to its added cost, and it shows no clear advantage over compute-matched random search. I have raised my score to 2, but I still lean toward rejection. I encourage the authors to revise the paper in light of the reviewers’ feedback.
The controller captures a meaningful share of the available revision gain, and its benefit is concentrated in the runs that trigger revision. Thank you for reconsidering our submission. We would like to address this point directly and specify how the feedback from all three reviewers has shaped the revision.
The NPO and SimNPO increments recover a substantial fraction of the improvement available from the tested local revisions. At the same decision checkpoint, evaluating every candidate in the frozen family yields mean gains of only +0.008024 and +0.014868 J over the fixed continuation. DIR-R obtains +0.002433 and +0.008228 J, capturing 30.32% and 55.34% of those available gains. This explains the scale of the increments: even the best branch in this tested family offers limited additional improvement. The measured runtime increases are 33% and 52%; we will present these rows as limited revision opportunities together with their explicit costs.
The GradDiff ablation shows why the distribution of gains also matters. In the primary four-field evaluation, six of twelve runs trigger revision. Those six have a mean gain of +0.3292 J and median gain of +0.3650 J; the other six keep the fixed continuation. Across all twelve, the mean is +0.1646 J and the median is +0.0025 J. The gate captures 58.26% of the available single-checkpoint gain at 1.45× the fixed trajectory's normalized update count. The gain is concentrated in the triggered runs, which we will show alongside the all-run mean and median. These are continuation-selection utility results.
The full-method baseline gains provide separate evidence for the overall approach. On LACUNA, the six direct comparisons improve mean utility by +0.4308–0.8479 J under NPO and +0.5034–0.5708 J under SimNPO, with a majority of paired wins in every comparison. The equal-cost random-search control addresses the replacement ranking and gives no overall advantage for DIR-R; we retain that result alongside the gate analysis.
We have already incorporated feedback from all three reviewers into the manuscript:
- zAGL: Section 5.6 now provides fixed-interface evidence for Intervention Score through the direct Natural-TOFU comparisons in Table S1 and the collateral-damage analysis in Table S2. This addresses the request for evidence supporting initial intervention selection.
- NaAs: The added discussion in §5.6 distinguishes the full DIR-R comparisons, fixed-selector comparisons and same-start controller ablation. This addresses the question of what the different experimental panels measure.
- ScPL: Section 5.6 adds direct Natural-TOFU comparisons, a Qwen3.5-4B initial-selection experiment and Figure 1 showing paired full-method gains and win counts, addressing your requests for direct evidence, newer-model coverage and clearer presentation.
We will definitely incorporate the remaining feedback from all three reviewers into the next manuscript revision. In particular:
- For zAGL, place the statistical intervals for the controlled result and the scope of the fallback rule beside the corresponding claims.
- For NaAs, make the baseline-screening and final-evaluation protocols explicit beside the comparison panels, and discuss each controller increment together with its available gain and cost.
- For ScPL, lead with direct paired comparisons, label the original 19/20 as a cross-query estimate, and identify branch-selection versus held-out scoring beside each relevant result.
We hope this clarifies both why some increments are small and where revision produces a meaningful benefit. We would be grateful if you could consider the direct evidence and these concrete revision commitments in reassessing the paper.
Controller ablation. Available gain is the best improvement in the tested candidate family at the same frozen checkpoint. Capture fractions use unrounded values.
| Objective | DIR-R gain | Available gain | Captured | Cost vs fixed continuation |
|---|---|---|---|---|
| NPO | +0.0024 | +0.0080 | 30.32% | 1.33× measured time |
| SimNPO | +0.0082 | +0.0149 | 55.34% | 1.52× measured time |
| GradDiff, four fields | +0.1646 | +0.2825 | 58.26% | 1.45× normalized updates |
Response to Reviewer ScPL
W1 — Direct comparisons
The concern applies to the combined Natural-TOFU estimate; direct comparisons support full DIR-R on LACUNA and Intervention Score on Natural-TOFU. All six LACUNA mean gaps favor full DIR-R, with field, seed, objective, budget and evaluator held fixed in each pair (R10). Natural-TOFU's 19/20 adds a revision gain from other queries, shifting SimNPO vs. GRAIL from −0.0044 to +0.0007 J. Q2 gives the direct selector comparisons.
W2 — Effect size and cost
The four-field ranking is primary, and the gate's practical benefit is lower search cost. From the same starting columns, revision adds +0.1646 J (6 wins, 6 ties; median +0.0025) at 1.45× the fixed-support updates versus 16× for exhaustive search. Five selectors have higher four-field means. Without Birth City, DIR-R leads Projected Causal by +0.0144 J on the original seeds; the ordering reverses on fresh seeds. R5 gives the smaller NPO and SimNPO gains and their run times.
W3 — Score and controller
DIR-R compares continuations from the current checkpoint, so it can change the edited columns without rescoring them. The initial score rewards predicted forgetting, deducts the larger positive retain or protected effect, and scales by neutral change. Candidates share batches and randomness. Q5 tests ranked swaps against random search; Q4 explains the ratio.
W4 — Newer models
We ran a small-scale experiment on the newer Qwen family. On Qwen3.5-4B, we use one Natural-TOFU author, three seeds, 25 steps and 5,998,080 editable scalars per method. Intervention Score reaches J = 0.0703 under NPO, beating WAGLE, Storage and Activation by +0.0299, +0.0195 and +0.0156 J (3/3 each). The order reverses under SimNPO (rebuttal Table R14). We plan to run the same comparison on Gemma 4. If the runs finish in time, we hope to post the results before the November 18 AoE rebuttal deadline.
W5 — Presentation
The revision plots full DIR-R's gains and explains what drives the Natural-TOFU selector gains. Figure 1 labels paired wins; S1/S2 separate utility from collateral damage. Original tables display four decimals. Q6 reports the actual swaps.
Q1 — Independent test
The new Qwen test scores on held-out queries; the GradDiff search comparison scores on its selection queries. With frozen thresholds, we ran four fresh seeds per setting at 3M and 6M editable scalars. Locator queries select the branch; separate forget, retain and protected queries score both continuations. Revision yielded 27 ties, two gains and three declines in 32 held-out pairs. R13 reports the five GradDiff selection scores.
Q2 — Natural-TOFU controls
The direct Natural-TOFU comparison favors Intervention Score over WAGLE and MemFlex when we vary only the ranking. R7 pairs runs by model, author and seed, with the same loss, down-projection space, 25 steps and about 6M editable scalars. The gaps are +0.1344/+0.2926 J over WAGLE (NPO/SimNPO, 94/120 wins each) and +0.0928/+0.2051 J over MemFlex (92/120 each). Lower collateral damage accounts for these gains and the gains against Storage (S2).
Q3 — LACUNA controls
The full-method panel tests the strongest screened baselines for each objective; the ablation tests the contribution of revision alone. We locked the top three by four-field mean J per objective, then used fresh seeds. Table 14 gives +0.4308 J vs. Projected Causal (8/12) under NPO and +0.5034 J vs. Activation-Down (12/12) under SimNPO (R10). Both arms in Table 15 start from the same columns (G1).
Q4 — Diagnostics and scaling
The denominator downweights beneficial edits that also shift neutral answers; separate loss rescaling can alter the ranking. A common positive scale c changes only the effective floor to ε/c. We use mean gold-token NLL and ε=10⁻⁸; the floor matters near zero neutral effect. See Revision Change Summary for the loss definitions and overlap.
Q5 — Random search and cost
Birth City drives the mean difference between DIR-R and random search at equal update counts and parameter coverage. From the same step-160 checkpoint, both searches use 80 or 200 updates per pair; stage times differ by less than 0.55%. The full-panel mean difference in selection score is −0.5530 J. Excluding Birth City gives DIR-R +0.0100 J across the two remaining pairs (one win, one loss; R13).
Q6 — Actual swaps
Qwen extends model coverage, and DIR-R changes which columns it edits at a fixed per-step budget. At step 160, DIR-R swaps 11,154 of 111,538 columns on Birth City, Email Address and Phone Number at seed 4099. The other two retain the original columns. Swapping keeps the number edited per step fixed and increases the total distinct scalars reached by about 10%. Figure 1 shows utility gains; these counts show actual support changes.