acceptodds
Withdrawn from ICLR 2027

Implicit Evocation in Unlearning-Based Alignment: Exploration and Explanation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language-model unlearning is commonly evaluated at the output, leaving open whether reduced target answering is accompanied by representational erasure. We study this question on RWKU using two instruction-tuned, approximately 4B-parameter models, three unlearning objectives (GA, NPO, and ATU), and both full-parameter and LoRA updates. Applying the model’s shared unembedding as a raw logit lens, we find that target-answer subtokens can remain more highly ranked than task-irrelevant baseline subtokens at intermediate layers even when their final-layer ranks decrease. We call this tested pattern implicit evocation. The observation establishes residual linear decodability under an oracle target-subtoken readout; it does not by itself establish attack-level recoverability or universality across unlearning methods. Under an idealized residual-block model and explicit boundedness and contractivity assumptions, we also derive conditions that favor stronger late-layer updates. This conditional analysis is consistent with late-layer suppression but does not fully explain the observed non-monotone mid-layer peak.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model, unlearning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_cuYCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper examines intermediate-layer target-answer signals after unlearning using a raw logit lens on two models and three unlearning objectives. It reports residual subtoken decodability despite output suppression and proposes a conditional analysis of layer-wise update intensity.

Critical strengths

The paper addresses the useful distinction between output suppression and internal accessibility, while appropriately limiting its claims about answer recovery.

Critical weaknesses
  1. The theoretical premise is infeasible in the stated model. LayerNorm’s shift invariance implies Jc1=0J_c\mathbf{1}=0, hence (I+WJc)1=1(I+WJ_c)\mathbf{1}=\mathbf{1} and ∥I+WJc∥2≥1\|I+WJ_c\|_2\geq1. This contradicts the strict contraction required by Theorem 3.2, leaving the proposed explanation unsupported within its own model.

  2. The empirical contribution is insufficiently established. Intermediate-layer target-token retention has already been reported by Wang et al. [1], which is not discussed. The best-subtoken statistic and unmatched irrelevant-token baseline do not adequately distinguish the reported phenomenon from readout and token-selection effects.

  3. The presentation makes verification unnecessarily difficult. Several plots have small labels, and figure terminology is inconsistent with the text, including “Unaligned” for pre-unlearning checkpoints. The figures and tables should be legible and consistently labeled without requiring explanatory corrections in the surrounding prose.

  4. Limited model coverage. The experiments are restricted to Phi-3 and Qwen3 at approximately 4B parameters. They do not establish whether the observed layer-wise pattern persists in newer architectures, such as the hybrid Gated DeltaNet–attention architecture of Qwen3.5. Testing a comparably sized model such as Qwen3.5-4B-Base would help distinguish a broader phenomenon from behavior specific to the evaluated checkpoints.

References

[1] Wang et al. Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models. ACL 2026. Paper

Questions for authors
  1. How is Theorem 3.2’s strict contraction assumption compatible with LayerNorm’s shift invariance?

  2. What substantive finding goes beyond Wang et al. [1], and does it survive controls matched for token frequency and candidate-set size?

  3. Does the observed pattern persist in a contemporary architecture, such as Qwen3.5-4B-Base, under the same controlled diagnostic?

I would consider raising the rating to 2 (Weak Rejection) if the authors resolve the incompatibility in the contraction assumption, clarify the contribution beyond prior work, and show that the empirical pattern survives matched-token controls. Validation on a contemporary architecture would further strengthen the evidence.

Reviewer_MsxjOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper try to find out whether output-level forgetting in LLM unlearning entails representational erasure. They use a raw logit lens on RWKU benchmark with two 4B models, three unlearning objectives, and full-parameter and LoRA updates, it finds that target-answer subtokens remain above irrelevant baselines at intermediate layers despite lower final-layer ranks. The authors call this “implicit evocation” and interpret it as residual linear decodability. A conditional theoretical analysis predicts larger updates in later layers. The paper also reports forgetting and utility metrics.

Critical strengths
  1. The authors explore an interesting question: the difference between "not answering in the output" and "true erasure from the representation." This question is important for the future unlearning community and deserves attention.
  2. This paper provides with details limitations.
Critical weaknesses
  1. Novelty of the experimental findings. Does current finding some kinds of overlap with the findings of prior work without proper citation? For example, the Head Projection attack proposed by Patil et al. (ICLR 2024) [1] applies the logit lens to intermediate layers and takes the top-k tokens from each layer to recover “deleted” facts.

  2. Validity of the experimental results and correctness of the metrics. The paper lacks a never-known control, i.e., how highly the answer tokens of facts the model never knew are ranked in the intermediate layers. Without this control, it is hard to determine whether the elevation in the intermediate layers is anomalous. Although the authors acknowledge this in the limitations, it is an important reference for validating the conclusion. Furthermore, for the curves in Figure 1, the authors provide a comparison for only a single prompt, and a conclusion cannot be drawn from a single example alone. Could the authors provide a cross-benchmark experimental design and quantitative statistics of the experimental results?

  3. Incomplete experimental results. The main table, Table 1, is missing the most important result: the performance of the LLM before unlearning. This directly makes it impossible to compare the effectiveness of the unlearning algorithms, which means it is unknown whether effective unlearning has been achieved, and the corresponding unlearning strength is also unknown. Combined with the experiments config in the appendix, the learning rate of GA and NPO is 6×10⁻⁸, which is extremely small and is very likely to lead to very poor unlearning.

  4. The theory is somewhat weak. The paper does not establish a connection between “update magnitude” and “decodability.” Theorems 3.1 and 3.2 concern gradients and update magnitudes, whereas the conclusion concerns logit-lens ranks.

[1] Patil, Vaidehi Ramesh, Peter Hase, and Mohit Bansal. "Can sensitive information be deleted from llms? objectives for defending against extraction attacks." International Conference on Learning Representations. Vol. 2024. 2024.

Questions for authors
  1. Could the authors explain whether their findings are, to some extent, consistent with those of Patil et al. (ICLR 2024)? If not, what are the main differences?

  2. Could the authors provide a never-known control experiments? In addition, could they provide a quantitative, cross-benchmark analysis of the curves in Figure 1?

  3. Complete experimental results are needed, especially the performance of the model before unlearning, so that the effect of unlearning can be compared.

  4. Could the authors provide a theory that connects “update magnitude” and “decodability”?

Reviewer_LpCfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores the possibility that unlearning at the output level deletes information about the target. The authors find that using a raw logit lens diagnostic (shared unembedding applied to hidden states), they observe that on RWKU for two 4B instruction tuned models (Phi-3 & Qwen3) and three unlearning objectives (GA, NPO, ATU), the target answer tokens are linearly decodable at intermediate layers but suppressed at the last layer (elicited by an implicit evocation). They also perform a conditional analysis for an idealized residual block model where there is a bias toward stronger updates at the late layer, which explains why the suppression at the end of the late layer is augmented, although the mid-layer peak is not explained by the apparent late-layer peak.

Critical strengths
  • Naming and formalizing “implicit evocation” focuses on the gap between the output suppression and the representational decodability in unlearning, underexplored and important distinction.
  • Use of raw logit lens as a zero-parameter diagnostic across layers is simple, reproducible, and insightful for the probing residual linear decodability without any training additional probes.
  • Conditional theory clarifies the sufficient conditions for gradient decay and late-layer update dominance in a stylized residual setting; this separation of assumptions and claims is useful.
Critical weaknesses
  • The paper theoretical results rely on strong, unverified assumptions (frozen attention, mean-centered LayerNorm while models uses RMSNorm and explicit contraction of I + WJc) and do not account for the non-monotone peak in the mid-layer observed experimentally.
  • The analysis provides a plausible rationale for late-layer suppression but cannot be tied to the specific diagnostic behavior (mid-stack elevation) measured in the experiments.
  • Some of the implementation details like random seeds, number of runs, sample sizes per plot are hard to find, limiting reproducibility and uncertainty quantification.
  • Forget-side pre-unlearning baselines are not reported, which complicates interpretation of output-level gains.
  • No matched never-known controls or retain-model baselines. Therefore, it is unclear how significant the mid-layer elevations are relative to recall under logit lens.
  • No representation targeting unlearning baselines (e.g. RMU/Adaptive RMU, PARS with generative probes, SAE-unlearn, or layer-selective FOM-UL) are included to see if the implicit evocation diminishes with methods that explicitly target internal states.
Questions for authors
  • How sensitive are the results to the candidate subtoken set design? Please report the average set sizes and ablate max vs. mean/first-token aggregation and stricter surface form rules.
  • Do the findings persist under calibrated readouts (e.g., Tuned Lens) or simple trained linear probes? How well do the tuned lens trajectories work compared to the raw logit lens?
  • Can you add matched never known entity controls to quantify how unusual the mid-layer elevations are relative to generic recall?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.