- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Knowledge Lens, a framework that casts each (question, candidate) pair as a True/False claim and reads two scores from a single forward pass: the True/False logit margin (external) and a cross-validated logistic-regression probe on the hidden state (internal). Both scores are converted to pairwise measures, K_ext and K_int, and their gap K_int − K_ext is averaged into “Suppressed Knowledge” (SK). The framework is applied to released unlearned models (8 LLM-GAT methods × 8 checkpoints, 3 RMU architectures, and 9 OPTML models, on WMDP-Bio and WMDP-Cyber). K_int stays at 0.68–0.76 while K_ext falls to 0.53–0.64 and MCQ accuracy falls to near chance. The authors conclude that behavioral forgetting overstates knowledge removal.
Critical strengths
- The question is important, and the research direction is highly relevant to the LLM unlearning community.
- The four-state item decomposition (Table 1) is a useful way to report unlearning results.
- The paper follows good reproducibility practices. Probing is out-of-fold, with question-level folds and per-fold layer and regularization selection. The appendix is detailed, the authors verify that the answer-token choice does not inflate the gap, and code is provided in the supplementary material.
Critical weaknesses
-
Missing related work. The related work omits the reinforcement-learning-based unlearning family (RULE [1], PURGE [2]) and the self-distillation family (UNDIAL [3], DUET [4], SHRED [5]). The discussion of unlearning evaluation also omits established benchmarks such as TOFU [6], MUSE [7], and RWKU [8].
-
Limited evaluation scope. The evaluation covers few models and method families, and uses a single benchmark setup (WMDP). This is not enough to support the general claim that unlearning should be evaluated through internal approaches.
-
Presentation.
- The term “unreal unlearning” does not convey the phenomenon it names and is confusing. A more descriptive name would help.
- Figure 1 appears to be AI-generated (the AI Use Statement discloses figure generation) and is sloppy. The hidden-layer magnifying glass is off-center and, for no apparent reason, connected to the output magnifying lens. The “correct answer” label overlaps with other elements. The figure is also not a pdf, so quality degrades when zooming in.
- The Figure 4 caption does not describe the plot. It does not say what evidence is shown or what it is evidence for.
-
The prompt is not shown to be debiased. Appendix A calls the prompt “debiased,” but the only mitigation is the sentence “True and False are equally likely.” Stating this in the prompt does not show that the model follows it, which is exactly what a True-bias is. The authors should refine this.
-
The 60.8 pp headline conflates metrics. It compares 4-way MCQ accuracy (5.8% of above-chance signal retained) with a pairwise ranking metric that gives partial credit (66.7%). Chance-to-base normalization does not make the two equivalent. K_ext itself retains 32.1% on the same scale, so roughly 26 pp of the 60.8 pp comes from the change in metric and format, not from the internal-vs-external distinction. The like-for-like gap is about 35 pp. In addition, accuracies are taken from other papers rather than recomputed in this pipeline, so behavioral forgetting is not measured in the same format as K_ext.
-
No validity evidence for SK. The paper never shows that SK predicts anything operational, even though LLM-GAT already reports tampering and relearning recovery for these same 64 checkpoints. The claim that the joint readout “distinguishes among methods” is asserted, not demonstrated. The abstract’s stronger claim, that unlearning “should be evaluated through internal, interpretable approaches,” outruns the evidence. A supervised probe trained with forget-domain labels is not obviously interpretable, and linear decodability does not imply that an attacker could extract the knowledge.
Minor Weaknesses:
- Line 276 is awkwardly phrased: “computed out of fold using the five-fold probing protocol.”
- Line 426: “the same” should be “similar.”
- Method order differs across Figure 2, Figure 3, and Table 1.
References: [1] RULE: Reinforcement UnLEarning Achieves Forget-retain Pareto Optimality, Zhang et al. NeurIPS 2025
[2] Reinforcement Unlearning via Group Relative Policy Optimization, Zaradoukas et al. ICLR 2026
[3] Undial: Self-distillation with adjusted logits for robust unlearning in large language models, Dong et al. ACL 2025
[4] DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teacher, Zhong et al. ICLR 2026
[5] SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion, Hu et al. NeurIPS 2027
[6] TOFU: A Task of Fictitious Unlearning for LLMs, Maini et al. COLM 2024
[7] MUSE: Machine Unlearning Six-Way Evaluation for Language Models, Shi et al. ICLR 2025
[8] RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models, Jin et al. NeurIPS 2024
Questions for authors
- Do K_int or SK correlate with recovered accuracy under tampering or relearning attacks?
- Does the joint readout separate checkpoints of equal accuracy in a way that matches downstream risk?
4 comments
Response to Reviewer 7qKV (2/2)
4. The 60.8 pp headline conflates metrics. Response: We agree on all three points, and the 60.8-point headline is removed. The abstract and the introduction now give the three retentions side by side and no longer subtract across formats. Section 4.2 now states that WMDP-Bio accuracy and the external score differ in question format and scoring, whereas the external and internal scores differ only in the readout. The difference between these two matched readouts is the one the reviewer computed, 34.6 points. If we instead score both readouts all-or-nothing, as accuracy is scored, the difference is 34.0 points. On the borrowed accuracies (0.70 before unlearning and 0.24 to 0.32 after, from Che et al.), we measured multiple-choice accuracy in our own pipeline, on the same 1,273 questions and checkpoints. We obtain 0.68 and 0.24 to 0.34. This accuracy retains 11.0% of its above-chance level before unlearning. The abstract and the introduction now use this value, and Section 4.2 reports both. The same output scored as True/False claims in the same all-or-nothing way retains 28.7%, and partial credit adds 3.4 points. The step the reviewer attributes to metric and format is therefore mostly the question format. The new appendix "Matched Comparison of the Two Readouts" reports these values.
5. No validity evidence for SK.
- (a) Predictive validity, and Q1. Response: We agree. The submitted version has no such test, and "distinguishes among methods" was asserted. The revision adds a new appendix, "Suppressed Knowledge and Recovery", and a short Section 4.8. At the level of methods, none of SK, the internal score or the external score correlates with LLM-GAT's published tampering or input-space attacks (eight methods, all |ρ| ≤ 0.52), and we report this as a null result. This test is weak: it has 8 methods, and after their strongest tampering attack 7 of them return to an accuracy of 0.57 to 0.71 (TAR stays at 0.36). We therefore test single questions. For each model, we take the questions the unlearned model answers wrongly and its base model answered correctly, and correlate each question's internal score with whether an attack recovers it. We ran two attacks with their authors' code on 18 unlearned models. The relearning attack of Fan et al. (ICML 2025) fine-tunes on documents and never sees a benchmark question. A higher internal score predicts recovery in 13 of the 18 models. In 4 of the 5 exceptions the attack does not succeed (accuracy changes by at most 0.05). The Retraining on T (RTT) attack of Deeb & Roger (2024) raises the accuracy of all 18 models. A higher internal score predicts recovery in 17 of them (Spearman ρ 0.17 to 0.40).
- (b) Q2: equal-accuracy checkpoints and downstream risk. Response: The same appendix tests this. Within a model the readout does track risk, as in (a): it tells which questions an attack recovers. Across models, our data do not allow this test (paragraph "Checkpoint level"). The 18 models differ in their external score (0.46 to 0.64), so groups with a similar score and the same base model are small, and the largest has seven models, nearly the same set as the eight methods in (a), where we find no correlation.
- (c) The abstract's claim. Response: We agree, and the sentence is rewritten. The probe is supervised and we do not interpret its weights, so "interpretable" is removed. "Should be evaluated through internal approaches" is replaced by what the evidence supports: internal evidence complements behavioral evaluation. On extraction, point (a) above gives the test we added, with two relearning attacks. In the abstract, the two sentences before the rewritten one no longer say "consistently" and "substantial".
6. Minor points.
- (a) Line 276. Response: We agree. The sentence now reads "All internal scores are out-of-fold (Section 3.1): no question is used to fit, or to select the layer and regularization of, the probe that scores it."
- (b) Line 426. Response: We agree. The sentence now reads "A similar internal-external separation appears on WMDP-Cyber." We made the same correction where Sections 4.5 and 4.6 summarize these results.
- (c) Method order. Response: We agree. Figures 2 and 3 now list the methods in the order of Table 1 and Section 4.1 (GradDiff, RMU, RMU-LAT, RepNoise, ELM, RR, TAR, PB&J), as do all other figures and tables.
We thank the reviewer again for the questions, which led to the new controls and analyses above. We believe these revisions address the concerns raised. We hope the reviewer will consider strengthening their support in light of them.
Thank you for the clarifications (although at the end of the rebuttal period). I lowered the rating to 1, because you had not responded to any of the reviews, in order to help the AC. Overall, I think the paper is rushed and needs more work, so I will maintain my original rating of 2.
Thank you for reading our responses and for restoring the original score. We submitted our responses late in the discussion period because we addressed the review with new experiments, and performing and adding them to the paper took time. If any points from your review still require further work after the revision, we would be grateful to know which ones. If the main concerns are resolved, we hope you will consider strengthening your support for the paper.
Response to Reviewer 7qKV (1/2)
We thank the reviewer for the detailed review and for recognizing the four-state item decomposition as "a useful way to report unlearning results".
1. Missing related work. Response: We agree, and all eight are now cited in Section 2. The methods paragraph adds the reinforcement-learning family (RULE, PURGE) and the self-distillation family (UNDIAL, DUET, SHRED). The evaluation paragraph now names the benchmarks by what they ask the model to forget (WMDP, TOFU, MUSE, RWKU). TOFU is also evaluated in the revision (Section 4.7).
2. Limited evaluation scope. Response: We agree that the submission used one benchmark family and that its general claim went beyond that scope. The revision adds breadth and narrows the claim. It adds a second benchmark, TOFU, with sixteen unlearned models on Llama-3.2-1B (Section 4.7), and new tests on the existing models (relearning attacks, probes trained outside WMDP-Bio). The paper now evaluates 92 unlearned models on five base models and three datasets. Breadth does not make the result uniform, and the paper now says so. Section 4.2 reports that the internal evidence beyond the answer text is clear for some methods and small for others, and the abstract no longer makes the general claim (point 5c). All our tasks select among fixed answer candidates, and the Conclusion states this limit.
3. Presentation.
(Continued in the next comment.)