- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how the distinction between learning-based and memorization-based generation affects the difficulty of machine unlearning. The authors develop a theoretical framework to explain why relabeling-based unlearning may be more challenging when model predictions rely on learned general rules, rather than memorized input-specific associations.
Through controlled binary classification and TOFU-LM experiments, they evaluate several unlearning methods and show that learning-based generation generally requires more optimization steps and causes greater temporary degradation in retain accuracy. Additional experiments examine the effects of different forget-set sizes, model architectures, and held-out paraphrases.
Overall, the paper explores an interesting and meaningful research question and provides useful theoretical and empirical insights into the mechanisms underlying machine unlearning. However, some aspects of the experimental design, particularly the use of replacement labels, warrant further examination to better distinguish the difficulty of forgetting existing knowledge from that of learning contradictory responses.
Critical strengths
-
Interesting and meaningful research question. The paper investigates an important yet relatively underexplored aspect of machine unlearning: how the underlying generation mechanism (learning-based vs. memorization-based) affects unlearning difficulty. This distinction provides a useful perspective for understanding why certain knowledge may be harder to forget than others. The research question is well motivated and has potential implications for developing more effective unlearning methods.
-
Combination of theoretical analysis and empirical validation. The paper provides a theoretical framework to explain the different behaviors of learning-based and memorization-based generation during unlearning. The authors further investigate these differences through controlled binary classification and TOFU-LM experiments, considering multiple unlearning methods and additional experimental settings. The combination of theoretical insights and empirical observations strengthens the paper’s contributions and provides a useful foundation for further research.
Critical weaknesses
While the paper presents an interesting and valuable perspective on machine unlearning, I have several concerns regarding the experimental design and the interpretation of the results.
-
The paper distinguishes learning-based and memorization-based generation using artificially constructed tasks (LINEAR/RANDOM and TOFU-LM). However, real-world LLM knowledge often involves both mechanisms simultaneously, making it unclear whether the findings generalize to realistic scenarios.
-
The experiments primarily implement unlearning by replacing original answers with incorrect labels. Consequently, the observed difficulty may reflect not only forgetting existing knowledge but also learning contradictory responses, particularly when the original predictions follow a shared rule.
-
Training models to produce incorrect answers may not be desirable in practical unlearning applications, where maintaining reliability and correctness is important. It would be valuable to investigate whether the observed differences persist when using neutral responses, such as “I don’t know,” instead of incorrect labels.
-
The evaluation primarily measures whether models stop producing the original answers, which does not necessarily indicate that the underlying knowledge has been removed. This concern is particularly relevant to TOFU-LM, where evaluation is restricted to four candidate answers.
-
Although the theoretical analysis offers an interesting explanation for the observed differences, the experiments do not directly establish that the slower convergence of learning-based unlearning results from the proposed hypothesis-space expansion. Further empirical evidence would help strengthen the connection between the theoretical analysis and the observed behavior.
Questions for authors
-
How well do the authors expect the learning-based and memorization-based distinction to generalize to real-world LLM knowledge, where both mechanisms may coexist? Is there any additional evidence supporting the applicability of the findings beyond the artificially constructed tasks?
-
Would the observed differences in unlearning difficulty persist if incorrect replacement labels were replaced with neutral responses, such as “I don’t know”? This comparison would help clarify whether the results primarily reflect knowledge forgetting or the difficulty of learning contradictory responses.
-
How do the authors distinguish genuine knowledge removal from response suppression in their experiments? In particular, could the original knowledge remain accessible through alternative prompts or evaluation strategies beyond the four-choice setting used in TOFU-LM?
-
Is there any empirical evidence directly supporting the proposed hypothesis-space expansion as the explanation for slower convergence in learning-based unlearning? Could alternative factors, such as differences in optimization difficulty or gradient interference, also explain the observed behavior?
2 comments
Maintain weak accept
The authors' response addresses most of my concerns, I maintain my weak accept recommendation.
We thank the reviewer for the positive assessment and the constructive suggestions.
Generalization to real-world knowledge (W1, Q1), real knowledge often involves both mechanisms. Our framework handles this with a mixture over hypotheses (Section 3.1). In a mixture experiment on Qwen3-4B, half of the input space follows a rule and the other half has random labels, and the results fall between the two pure tasks (Appendix C.7.5). Existing unlearning benchmarks do not mark which facts were learned as rules and which were memorized, so they cannot isolate the effect we study. Extending the study to natural data is future work.
Incorrect labels versus neutral responses (W2, W3, Q2), thank you for suggesting this comparison. We ran it on TOFU-LM, replacing the incorrect forget targets with "I don't know" (new Appendix C.6). During scoring we add "I don't know" as a candidate, so a retain question answered with it counts as an error. The gap remains. In the learning-based condition, retain accuracy drops further and recovers later, original answers are suppressed more slowly, and joint convergence takes nearly twice as long as in the memorization-based condition. Two other results agree. NPO, which uses no replacement targets at all, also suppresses original answers more slowly in the learning-based conditions (Table 2). In the new BAND control (W5, Q4), the model fits contradictory labels and still unlearns as fast as on the memorization-based task. Together, these results suggest that the difficulty comes from fitting new responses while a learned rule predicts the old ones.
Suppression versus removal (W4, Q3), we measure unlearning performance primarily with accuracy on the forget and retain sets, a metric widely used in prior LLM unlearning work (e.g., Li et al., 2024; Deeb & Roger, 2024). To test whether suppression generalizes beyond the prompts used during unlearning, we follow Wu et al. (2025) and evaluate held-out paraphrases of each forget question, which were excluded from both fine-tuning and unlearning (Section 4.2). The paraphrases ask for the same attribute in different words, so they show whether the original answer is still reachable from other prompts. In the learning-based condition, held-out accuracy stays elevated after accuracy on the training prompts has approached zero, and held-out unlearning is much slower than in the memorization-based condition (Table 2). This fits our main finding. Complete or certified removal of the underlying knowledge remains an ongoing area and is beyond the scope of this paper. We leave stronger evaluations, such as free generation, to future work.
Evidence for hypothesis-space expansion (W5, Q4), we have added controlled experiment on the binary task (new Appendix C.4). BAND and BAND-MIXED use the LINEAR rule, relabel the same number of points at a similar distance from the decision boundary, and differ only in whether the relabeled points can be fit by moving the boundary. When they can (BAND), unlearning converges about as fast as for memorized labels. When they can only be fit as exceptions (BAND-MIXED), unlearning is slower and retain error roughly doubles. The final decision boundaries on LINEAR show the original rule with exceptions at the forget points (Appendix C.7.1). These results support the view that unlearning slows down when the model has to move from its rule to a rule with exceptions.