- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies when forgetting is harder: removing the knowledge early in pretraining or unlearn than knowledge introduced later. The authors present theoretical analyses and controlled experiments on Pythia-410M and TinyLlama-1.1B, evaluating four unlearning methods using perplexity and membership-inference attacks.
Critical strengths
The paper's central question addresses an important limitation of post-training unlearning benchmarks. Training models with controlled forget-data introduction is more informative than evaluating only conventional fine-tuning settings.
Critical weaknesses
My specific issues with the paper are the following:
(1) The paper mainly measures the forgetting process in terms of forget-set perplexity and membership inference AUC. Higher perplexity does not necessarily mean good unlearning, since the model trained with retaining only has a forget-set perplexity of , and several other post-unlearning models have forget-set perplexities close to .
I highly recommend to use some other interpretable measures such as verbatim memorization (VerbMem), KnowMem, and accuracy on forget/retain sets. Note that the retain utility also needs to be measured against similar forgetting level.
(2) The theoretical analysis fails to prove the claimed difficulty of the unlearning process. The theorem only demonstrates that parameter distance of an unlearning approach from the retain-only method could be increased if the data was introduced early in the learning phase. This happens in a special case for a two-layer linear network with restrictive assumptions, and the parameter distance does not necessarily capture the difficulty (or unlearning cost) of the unlearning problem.
Questions for authors
Please check the Critical Weaknesses setion.
5 comments
Thank you for the detailed response and additional experiments. However, two concerns remain:
(1) Evaluation: VerbMem is a useful addition, but it primarily measures text reproduction rather than retained knowledge. Knowledge-based evaluations (e.g., KnowMem or QA accuracy) would provide stronger evidence.
(2) Theory: I appreciate the authors' clarification that parameter distance does not necessarily imply unlearning difficulty. However, this remains my central theoretical concern. The results establish parameter displacement under restrictive assumptions, rather than a direct relationship with unlearning cost or difficulty. Therefore, the theoretical contribution should be framed as providing intuition rather than establishing the claimed difficulty.
Overall, my main concerns remain only partially addressed. I will maintain my current rating.
Thank you for evaluating our responses and clarifying the remaining concerns.
(1) Evaluation: To complement VerbMem, we added the results for KnowMem and QA accuracy as requested by the reviewer. We used Claude Sonnet 5 to generate 40 multi-choice questions where the blanks are characters or place names used in the novel we use as the forget set. We used the recent larger Pythia 2.8B model to evaluate the average QA accuracy and KnowMem score [1] before and after unlearning with 15 epochs of SimNPO. As the following table shows, by decreasing the value of we get a higher accuracy and higher KnowMem score after unlearning by decreasing the value of and the values are higher for compared to finetuned models (despite comparable initial accuracies), which confirms that it is harder to unlearn for lower values of .
| Model | Acc (Before) | Acc (After Unlearning) | KnowMem |
|---|---|---|---|
| 66.3 | 50.8 | 7.3 | |
| 64.0 | 42.3 | 7.1 | |
| FT | 54.0 | 0.0 | 4.6 |
| FT | 62.3 | 3.0 | 5.8 |
| FT | 66.6 | 14.0 | 6.4 |
[1] Shi, W., Lee, J., Huang, Y., Malladi, S., Zhao, J., Holtzman, A., ... & Zhang, C. (2025, May). Muse: Machine unlearning six-way evaluation for language models. In International Conference on Learning Representations (Vol. 2025, pp. 27797-27818).
(2) Theory: To address this concern, we extended our theoretical analysis to establish a direct lower bound on the geometric unlearning cost defined in Definition 3.1, under the assumptions of our two-layer linear model and therefore close the gap in our theoretical analysis. The following file is a standalone file containing the theoretical results and the complete proofs for the new theorems:
We summarize the main results below.
1. Function-space monotonicity. We first show that the coordinatewise ordering underlying Corollary 3.8 also establishes an ordering in function space:
\geq \lVert f_{\tau_2}(x)-f_r(x)\rVert_2, \qquad \forall x,\quad \tau_1<\tau_2.$$ Unlike Euclidean parameter distance, this functional discrepancy is invariant to function-preserving reparameterizations. Thus, the stage-dependent ordering is not limited to parameter-space measurements. **2. Lower bound on geometric unlearning cost.** We establish a lower bound directly on the minimum feasible parameter displacement $C(\tau)$: $$C(\tau)\geq g_S\left((D_\tau-\varepsilon_r)_+\right), \qquad g_S(q)=\sqrt{2S+2q}-\sqrt{2S},$$ where $D_{\tau} = \lVert M_{\tau}^{\ast} - M_r^{\ast} \rVert_F$ denotes the end-to-end functional distance, $S$ is the maximum target singular value, and $\varepsilon_r = \rho_r + \sqrt{\rho_r^2 + \xi_r}$, with $\rho_r = \lVert M_r^{\ast} - A_r \rVert_F$. This result follows from the retain-loss constraint and applies to all parameter updates satisfying the unlearning constraints in Definition 3.1, without requiring the unlearned model to preserve the original balanced-factor structure. **3. Connecting the original parameter-distance theorem to unlearning cost.** Let $B_{\tau}$ denote the right-hand side of Equation (2) in Theorem 3.6, which establishes $\lVert \theta_{\tau}^{\ast} - \theta_r^{\ast} \rVert_F^2 \geq B_{\tau}$. Using the balanced-factor structure and coordinatewise feature bounds from the original theorem, we derive a lower bound $H_{\tau}$ on functional distance: $$D_\tau\geq H_\tau \quad\Longrightarrow\quad \boxed{C(\tau)\geq g_S\left((H_\tau-\varepsilon_r)_+\right).}$$ So, **when endpoint forget loss and retain-loss gap are matched across stages, this lower bound is non-increasing with $\tau$**. This establishes a direct theoretical connection between the training-history-dependent parameter displacement characterized by Theorem 3.6 and our defined geometric unlearning cost. Together, the new results establish the following theoretical progression: **Training history → Parameter displacement → Function-space displacement → Lower bound on geometric unlearning cost** Our additional experiments measuring optimizer steps, VerbMem, KnowMem, Q/A accuracy, forgetting–utility tradeoffs, and MIAs provide complementary evidence of practical unlearning difficulty across training histories. We will incorporate these theoretical results and empirical analyses into the revised manuscript to clarify the connection between our structural analysis and practical unlearning difficulty.
We thank the reviewer for acknowledging that our work reveals an important limitation of post-training unlearning benchmarks. We address the reviewer’s concerns in the following.
W1. New metric requested by the reviewer. Thank you for suggesting a generation-based evaluation of forgetting. We have added VerbMem [1] results in Table 3 measuring the mean ROUGE-L F1 overlap between generated continuations and the original forget text, scaled by 100.
The models begin with similar mean VerbMem scores (11.87 - 12.34). Across all four methods, post-unlearning VerbMem decreases as τ increases. Within each method, every evaluated FT model has lower residual VerbMem than every maintained-exposure model. These results extend the observed training-history dependence beyond perplexity: under the tested method-specific budgets, earlier maintained exposure leaves more residual reproduction of the forget set.
In addition to the metric requested by the reviewer we also provided more empirical evidence in our response to the reviewer xhRW on the following (please see our complete response for the results):
Optimization steps required for forgetting. Rather than relying on forget-set perplexity, we measure the number of optimizer steps required to reach a calibrated Min-K membership-inference AUC of 0.5, corresponding to the retain-only reference. The results show that earlier-maintained knowledge consistently requires more optimization steps
Forgetting–utility tradeoff. We further evaluated the forgetting–utility tradeoff by constructing a Pareto frontier over all tested learning rates and intermediate checkpoints, measuring retained utility using the change in WikiText-2 perplexity from each model's initial value.
[1] Huang, Jing, Diyi Yang, and Christopher Potts. "Demystifying verbatim memorization in large language models." Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.
[2] Shi, W., Lee, J., Huang, Y., Malladi, S., Zhao, J., Holtzman, A., ... & Zhang, C. (2025, May). Muse: Machine unlearning six-way evaluation for language models. In International Conference on Learning Representations (Vol. 2025, pp. 27797-27818).
W2: Theoretical results and restrictive assumptions.
We agree that a larger parameter distance from the retain-only model does not, by itself, imply greater unlearning difficulty. The purpose of our theoretical results is to characterize how training history shapes parameter displacement from the retain-only model through shared feature directions, rather than to establish that this displacement directly determines unlearning cost. Instead, we investigate this relationship empirically through several experiments spanning different models, unlearning methods, and evaluation metrics.
Regarding the assumptions underlying our theoretical analysis, we emphasize that they are grounded in recent prior work (Assumptions 3.3 and 3.4 are taken from Feng et al. (2026) and the small initialization follows Gidel et al. (2019)). As is common in the literature, our theoretical analysis is intended to provide mathematical intuition for the observed phenomena rather than to establish their validity in general LLMs. Importantly, our theoretical analysis complements, rather than substitutes for, the extensive empirical evidence presented in our work.
To further improve the empirical evidence on unlearning difficulty, we expanded our experiments to a larger model as well (Pythia 2.8B). As the following results show, we see the same trends for membership inference attacks and the forget-set perplexity.
Membership Inference figure for Pythia 2.8B: https://shorturl.at/ULYEs