- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies why catastrophic forgetting still occurs during SFT, even when techniques for protecting old knowledge are applied. The authors present an analysis that decomposes changes in old-task loss into four terms: a controlled term, probe mismatch, gradient staleness, and finite-step effects. They conduct controlled QLoRA experiments on constraint placement and replay timing. The results show that local constraints do not guarantee behavioral retention. Moreover, methods with similar final accuracy can have different forgetting patterns during training.
Critical strengths
The finding that applying the constraint before or after Adam matters a lot is very interesting. In Table 1, the experiment demonstrates a significant gap between the pre-Adam and post-Adam results in code adaptation (97.83% vs. 7.07% Legacy accuracy under aggregate constraints). This finding matters because, to the best of my knowledge, no previous work has discussed this choice, or shown that the placement of the constraint can make such a big difference.
Critical weaknesses
The connection between the analysis and the main experimental findings is very limited. The authors claim that explaining forgetting under constrained updates is their main contribution (end of Section 1). The loss decomposition could help explain why local constraints do not guarantee retention, but its connection with the main generation results is limited. For example, the full training runs show a big gap between pre-Adam and post-Adam constraints (Table 1), whereas the single-step experiments do not establish a clear advantage (Table 27). I understand that these experiments measure different outcomes, and that the authors distinguish local effects from long-term effects. However, it remains unclear which mechanism explains the long-term difference, and why the single-step experiments do not show it. Since this is an analysis paper, I think this gap weakens the support for its claimed explanatory contribution.
Questions for authors
-
The paper is hard to follow. In the main text, the authors use many terms that are hard to understand without explaining them. For example, in Section 2, the authors use the term "Legacy", but it is unclear what "Legacy" refers to. Other examples include "frozen Holm families" (Section 4.1), "structure–location factorial" (Section 4.1), and "open-web replay" (Section 4.1). I suggest that the authors use more accessible terms or explain them clearly.
-
The main text provides only limited discussion of how the paper is positioned relative to related fields, such as SFT and continual learning. Clarifying this would help readers better understand which gap the paper address. A short and clearly organized summary of the main contributions at the end of the introduction would also help readers understand what is new in this paper.
-
The organization of the experiments is hard to follow. Section 4.1 includes several separate studies, but it is not easy to keep track of which study each result comes from. For example, Section 4.2 discusses both the refresh–coverage study and the structure–location study, which use different experimental settings. It would be helpful if the authors could reorganize this part to help readers better understand the experimental section.
-
Regarding the critical weakness, it would be helpful if the authors could explain in more detail which mechanisms lead to the long-term retention gap between pre-Adam and post-Adam projection.
4 comments
W1 / Q4. The connection between the analysis and the main experimental findings is limited: which mechanism explains the long-run gap between pre- and post-Adam projection, and why do the single-step experiments not show it?
R4. The decomposition explains the gap, and the mechanism it points to, what Adam's first moment accumulates, is confirmed by a causal test.
Constraint-based protection removes the part of each update that would raise the loss on protected Old examples. Proposition 1 shows that this only keeps one term of the Old-loss change, the controlled term, from rising; the other terms keep accumulating unless the controlled term actively falls. Where the removal happens relative to Adam decides whether it falls (Table 1: 97.83% vs 7.07% of the Old answers kept). The decomposition locates the gap: moving the projection from before to after Adam raises the summed Old-loss change by +1.42 nats, and the controlled term carries +1.47 of it; probe mismatch and the finite-step remainder do not change. Every projected policy pays a similar uncontrolled cost. Pre-Adam offsets it by driving the controlled term to about −2.0; post-Adam cannot, because it only clips each step's controlled term at zero. The controlled term differs because of the optimizer state: Adam's first moment, its running average of past gradients, sets the direction of every step. Post-Adam projects only the finished step, so the raw, Old-harming gradient enters the first moment and the optimizer keeps proposing Old-harming steps that the projection can only trim. Pre-Adam projects each gradient before it enters Adam's moments, so the first moment accumulates the corrected gradient and the optimizer itself proposes steps that lower the Old loss.
A causal test. New runs keep post-Adam's exact constraint at every step and change only which gradient enters each Adam moment (seeds 63001–63004; the submitted runs, which the new platform reproduces exactly, are the controls):
| What Adam's moments receive | Controlled term | Legacy answers kept | Projection active |
|---|---|---|---|
| Raw gradient in both (post-Adam, submitted) | −0.53 | 6.5% | 76% |
| Projected gradient, second moment only | −0.33 | 5.4% | 81% |
| Projected gradient, first moment only | −1.97 | 94.6% | 11% |
| Projected gradient, both moments | −2.09 | 96.7% | 11% |
| Pre-Adam (submitted) | −2.01 | 96.7% | — |
The first moment carries the effect. Changing only the first moment turns post-Adam's failure into pre-Adam-level retention on every seed (+87 to +91 points), with a pre-Adam-like controlled term; the second moment alone does nothing. Update norms and New accuracy are matched. Why the single-step tests (Table 27 of the submission) show no gap. Their branches start every policy from one shared state (parameters, Adam moments, batch), which removes this mechanism by construction, so the controlled term is the same for all policies. On each policy's own trajectory the moments diverge and the difference accumulates to the 1.47-nat gap: Table 27 is the control that isolates the mechanism. The link to the generation results. Pre-Adam's restoring term is near zero while its Legacy accuracy falls to 29.9% and grows as accuracy recovers to 97.8%; post-Adam's never forms, and its answers do not return. Beyond Mistral-7B. The same signature holds in all 28 paired contrasts on four more models and under full-parameter SFT.
We have also revised the manuscript acoordingly. Please check Section 4.2; Appendix N.1–N.2.
Together, these changes make the explanatory chain explicit: the decomposition identifies the term that carries the gap, the optimizer state explains why that term differs, and the moment-source intervention confirms it causally. We hope this resolves the main concern, and we would welcome any further questions.
Response to Authors
Thank you for the detailed response and additional experiments. The new experiment on Adam’s first moment provides a clearer explanation of the retention gap between pre-Adam and post-Adam projection. The explanation of why the single-step experiments do not show the same gap is also helpful. These results address an important part of my original concern.
However, I still find the revised paper difficult to follow. The authors have added definitions, but the main text still uses many terms that readers need to remember and distinguish. For example, Sections 4.1–4.2 discuss Legacy reference-exact accuracy, independent strict-EOS accuracy, the full retention contract, and supporting New non-inferiority. It remains difficult to keep track of their different settings and understand which conclusion each result supports. Adding definitions and a glossary helps, but I think the main text still needs simpler explanations.
The organization has also improved with the research questions and separate subsections. However, the connection between the theoretical analysis, optimizer experiments, and generation results is still not sufficiently clear to me. The mechanism discussion in Section 4.2 contains many findings in one paragraph and refers to several appendix sections. It would help to explain this connection step by step, and clarify which conclusions follow from the analysis and which are supported by the additional experiments.
Overall, I acknowledge the improvements and the additional evidence. However, my concerns about the presentation and clarity of the explanation remain. I therefore maintain my original rating of 2 (weak rejection).
Thank you for the feedback. We have clarified the evaluation measures in Sections 4.1–4.2 and the connection between theory, optimizer interventions, and generation results, distinguishing theoretical conclusions from experimental findings. Please let us know if anything remains unclear; we welcome further questions.
Thank you for your review. We answer it with the decomposition itself and test the answer causally, and we have reorganized the paper so that each result is visibly tied to the analysis. We use W for each weakness, Q for each question, and R for our response. Section and appendix numbers refer to the revised PDF; table numbers refer to the submission.
Q1. Many terms are hard to understand without explanation (Legacy, frozen Holm families, structure–location factorial, open-web replay).
R1. Each term has a plain definition at first use, and the Appendix Guide adds a glossary of all terms. Legacy: the Old prompts the base model answered correctly before fine-tuning (23 for Mistral-7B). Legacy accuracy is the share still answered correctly at the end. Frozen Holm families: groups of statistical tests fixed before the confirmatory runs, with the Holm multiple-testing correction applied within each group. Structure–location factorial: the 2×2 experiment crossing the constraint's structure (one aggregate constraint vs one per Old-behavior category) with its location (projection before vs after Adam). Open-web replay: training on generic web text (OpenWebText) with a KL term, as in the ARR baseline, as opposed to replaying Old examples.
Q2. The positioning relative to SFT and continual learning is limited, and a clear summary of contributions would help.
R2. Prior work proposes protections and evaluates their endpoints; our paper explains, at the level of the realized optimizer update, why a protection that is satisfied at every step still forgets. Appendix K opens with a comparison table covering five lines of work: episodic gradient constraints (GEM, A-GEM, I-GEM), subspace projection (PEGP, Adam-NSCL, OPLoRA), regularization (EWC, CLoRA), replay and functional targets (ER, DER, FROMP, ARR, MSSR), and analyses of forgetting. For each it states what the method protects and what our analysis adds, for example where the constraint should act relative to Adam and how often a cached constraint should be refreshed. In the main text, the introduction closes with four contributions, introduced against these lines of work with pointers to Section 5 and the table: A functional-path account. Proposition 1 separates the term a constraint controls from three terms it leaves free, and a counterexample satisfies every cached constraint while the Old loss grows geometrically. Its staleness term forecasts the refresh interval a model needs, confirmed on 128 new runs. Placement decides retention. At equal step size, projecting before rather than after Adam keeps 97.83% versus 7.07% of the Old answers. The decomposition places the gap in the controlled term, a causal test traces it to Adam's first moment, and the signature holds on five models (7B–32B) and under full-parameter SFT. Paths versus endpoints. Pre-Adam projection and replay both end at 97.83%, yet 9 versus 152 of the 184 answers are correct at every checkpoint. Task-level evidence. Over 36 paired chains in all six task orders, replay keeps every earlier task in 35, whereas each constraint-based method loses one by more than 20 points in 10 to 29.
Q3. The organization of the experiments is hard to follow; Section 4.2 mixes the refresh–coverage and structure–location studies.
R3. Section 4.1 opens by stating five research questions, each with the study that answers it, and the two studies you mention have separate subsections. RQ1, where should a constraint act, and RQ2, are answers kept along the path or recovered at the end: the structure–location study (4.2, Table 1); RQ3, what does a satisfied constraint leave uncontrolled: the refresh–coverage study and the new refresh ladder (4.3); RQ4, when should replay targets arrive and what does replay protect: the replay schedules (4.4) and the structure–location study's Natural and Extraction tasks (4.2, Table 2); RQ5, which learned tasks survive a sequence: the sequential chains (4.5, Table 3).
All of the refresh–coverage study, including its summary of the 64 projection runs, now sits in 4.3, so 4.2 reports the structure–location study alone: its Code factorial (Table 1) and its two additional tasks (Table 2). The appendix opens with an Appendix Guide that maps every appendix to these questions.