acceptodds
Under review as a conference paper at ICLR 2027

Why Constrained Updates Still Forget: A Functional-Path Analysis of Continual SFT

Abstract

Supervised fine-tuning adapts language models to specialized tasks but can disrupt previously acquired behavior. Existing approaches constrain parameters, gradients, or replay objectives, yet the connection between the protected object, the intervening trajectory, and retained outputs remains unclear. We develop a functional-path account that decomposes old-loss changes along realized updates into the controlled direction, gradient staleness, probe mismatch, and finite-step effects. The account yields a conditional refresh budget and identifies the restoring direction supplied by informative replay. Matched QLoRA interventions separate constraint granularity from placement before or after Adam: placement strongly affects retention across the studied code-label audits, whereas the granularity interaction depends on the endpoint. Checkpointed generation separates endpoint recovery from higher sampled-path accuracy, and matched replay schedules reveal loss exposure concealed by final scores. A sequential-task comparison extends this distinction to backward transfer: subspace regularization and projection improve average retention while trading off different old tasks; supervised replay protects both more consistently in this setting. Additional response interfaces distinguish content gains from exact-response retention. These findings connect update-level protection to task-wise behavior and motivate mitigation through explicit functional anchors, trajectory evaluation, and joint old/new auditing.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Catastrophic Forgetting, Continual Learning, Large Language Models, Supervised Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rsrLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies why catastrophic forgetting still occurs during SFT, even when techniques for protecting old knowledge are applied. The authors present an analysis that decomposes changes in old-task loss into four terms: a controlled term, probe mismatch, gradient staleness, and finite-step effects. They conduct controlled QLoRA experiments on constraint placement and replay timing. The results show that local constraints do not guarantee behavioral retention. Moreover, methods with similar final accuracy can have different forgetting patterns during training.

Critical strengths

The finding that applying the constraint before or after Adam matters a lot is very interesting. In Table 1, the experiment demonstrates a significant gap between the pre-Adam and post-Adam results in code adaptation (97.83% vs. 7.07% Legacy accuracy under aggregate constraints). This finding matters because, to the best of my knowledge, no previous work has discussed this choice, or shown that the placement of the constraint can make such a big difference.

Critical weaknesses

The connection between the analysis and the main experimental findings is very limited. The authors claim that explaining forgetting under constrained updates is their main contribution (end of Section 1). The loss decomposition could help explain why local constraints do not guarantee retention, but its connection with the main generation results is limited. For example, the full training runs show a big gap between pre-Adam and post-Adam constraints (Table 1), whereas the single-step experiments do not establish a clear advantage (Table 27). I understand that these experiments measure different outcomes, and that the authors distinguish local effects from long-term effects. However, it remains unclear which mechanism explains the long-term difference, and why the single-step experiments do not show it. Since this is an analysis paper, I think this gap weakens the support for its claimed explanatory contribution.

Questions for authors
  1. The paper is hard to follow. In the main text, the authors use many terms that are hard to understand without explaining them. For example, in Section 2, the authors use the term "Legacy", but it is unclear what "Legacy" refers to. Other examples include "frozen Holm families" (Section 4.1), "structure–location factorial" (Section 4.1), and "open-web replay" (Section 4.1). I suggest that the authors use more accessible terms or explain them clearly.

  2. The main text provides only limited discussion of how the paper is positioned relative to related fields, such as SFT and continual learning. Clarifying this would help readers better understand which gap the paper address. A short and clearly organized summary of the main contributions at the end of the introduction would also help readers understand what is new in this paper.

  3. The organization of the experiments is hard to follow. Section 4.1 includes several separate studies, but it is not easy to keep track of which study each result comes from. For example, Section 4.2 discusses both the refresh–coverage study and the structure–location study, which use different experimental settings. It would be helpful if the authors could reorganize this part to help readers better understand the experimental section.

  4. Regarding the critical weakness, it would be helpful if the authors could explain in more detail which mechanisms lead to the long-term retention gap between pre-Adam and post-Adam projection.

4 comments

Thank you for your review. We answer it with the decomposition itself and test the answer causally, and we have reorganized the paper so that each result is visibly tied to the analysis. We use W for each weakness, Q for each question, and R for our response. Section and appendix numbers refer to the revised PDF; table numbers refer to the submission.

Q1. Many terms are hard to understand without explanation (Legacy, frozen Holm families, structure–location factorial, open-web replay).

R1. Each term has a plain definition at first use, and the Appendix Guide adds a glossary of all terms. Legacy: the Old prompts the base model answered correctly before fine-tuning (23 for Mistral-7B). Legacy accuracy is the share still answered correctly at the end. Frozen Holm families: groups of statistical tests fixed before the confirmatory runs, with the Holm multiple-testing correction applied within each group. Structure–location factorial: the 2×2 experiment crossing the constraint's structure (one aggregate constraint vs one per Old-behavior category) with its location (projection before vs after Adam). Open-web replay: training on generic web text (OpenWebText) with a KL term, as in the ARR baseline, as opposed to replaying Old examples.

Q2. The positioning relative to SFT and continual learning is limited, and a clear summary of contributions would help.

R2. Prior work proposes protections and evaluates their endpoints; our paper explains, at the level of the realized optimizer update, why a protection that is satisfied at every step still forgets. Appendix K opens with a comparison table covering five lines of work: episodic gradient constraints (GEM, A-GEM, I-GEM), subspace projection (PEGP, Adam-NSCL, OPLoRA), regularization (EWC, CLoRA), replay and functional targets (ER, DER, FROMP, ARR, MSSR), and analyses of forgetting. For each it states what the method protects and what our analysis adds, for example where the constraint should act relative to Adam and how often a cached constraint should be refreshed. In the main text, the introduction closes with four contributions, introduced against these lines of work with pointers to Section 5 and the table: A functional-path account. Proposition 1 separates the term a constraint controls from three terms it leaves free, and a counterexample satisfies every cached constraint while the Old loss grows geometrically. Its staleness term forecasts the refresh interval a model needs, confirmed on 128 new runs. Placement decides retention. At equal step size, projecting before rather than after Adam keeps 97.83% versus 7.07% of the Old answers. The decomposition places the gap in the controlled term, a causal test traces it to Adam's first moment, and the signature holds on five models (7B–32B) and under full-parameter SFT. Paths versus endpoints. Pre-Adam projection and replay both end at 97.83%, yet 9 versus 152 of the 184 answers are correct at every checkpoint. Task-level evidence. Over 36 paired chains in all six task orders, replay keeps every earlier task in 35, whereas each constraint-based method loses one by more than 20 points in 10 to 29.

Q3. The organization of the experiments is hard to follow; Section 4.2 mixes the refresh–coverage and structure–location studies.

R3. Section 4.1 opens by stating five research questions, each with the study that answers it, and the two studies you mention have separate subsections. RQ1, where should a constraint act, and RQ2, are answers kept along the path or recovered at the end: the structure–location study (4.2, Table 1); RQ3, what does a satisfied constraint leave uncontrolled: the refresh–coverage study and the new refresh ladder (4.3); RQ4, when should replay targets arrive and what does replay protect: the replay schedules (4.4) and the structure–location study's Natural and Extraction tasks (4.2, Table 2); RQ5, which learned tasks survive a sequence: the sequential chains (4.5, Table 3).

All of the refresh–coverage study, including its summary of the 64 projection runs, now sits in 4.3, so 4.2 reports the structure–location study alone: its Code factorial (Table 1) and its two additional tasks (Table 2). The appendix opens with an Appendix Guide that maps every appendix to these questions.

W1 / Q4. The connection between the analysis and the main experimental findings is limited: which mechanism explains the long-run gap between pre- and post-Adam projection, and why do the single-step experiments not show it?

R4. The decomposition explains the gap, and the mechanism it points to, what Adam's first moment accumulates, is confirmed by a causal test.

Constraint-based protection removes the part of each update that would raise the loss on protected Old examples. Proposition 1 shows that this only keeps one term of the Old-loss change, the controlled term, from rising; the other terms keep accumulating unless the controlled term actively falls. Where the removal happens relative to Adam decides whether it falls (Table 1: 97.83% vs 7.07% of the Old answers kept). The decomposition locates the gap: moving the projection from before to after Adam raises the summed Old-loss change by +1.42 nats, and the controlled term carries +1.47 of it; probe mismatch and the finite-step remainder do not change. Every projected policy pays a similar uncontrolled cost. Pre-Adam offsets it by driving the controlled term to about −2.0; post-Adam cannot, because it only clips each step's controlled term at zero. The controlled term differs because of the optimizer state: Adam's first moment, its running average of past gradients, sets the direction of every step. Post-Adam projects only the finished step, so the raw, Old-harming gradient enters the first moment and the optimizer keeps proposing Old-harming steps that the projection can only trim. Pre-Adam projects each gradient before it enters Adam's moments, so the first moment accumulates the corrected gradient and the optimizer itself proposes steps that lower the Old loss.

A causal test. New runs keep post-Adam's exact constraint at every step and change only which gradient enters each Adam moment (seeds 63001–63004; the submitted runs, which the new platform reproduces exactly, are the controls):

What Adam's moments receiveControlled termLegacy answers keptProjection active
Raw gradient in both (post-Adam, submitted)−0.536.5%76%
Projected gradient, second moment only−0.335.4%81%
Projected gradient, first moment only−1.9794.6%11%
Projected gradient, both moments−2.0996.7%11%
Pre-Adam (submitted)−2.0196.7%—

The first moment carries the effect. Changing only the first moment turns post-Adam's failure into pre-Adam-level retention on every seed (+87 to +91 points), with a pre-Adam-like controlled term; the second moment alone does nothing. Update norms and New accuracy are matched. Why the single-step tests (Table 27 of the submission) show no gap. Their branches start every policy from one shared state (parameters, Adam moments, batch), which removes this mechanism by construction, so the controlled term is the same for all policies. On each policy's own trajectory the moments diverge and the difference accumulates to the 1.47-nat gap: Table 27 is the control that isolates the mechanism. The link to the generation results. Pre-Adam's restoring term is near zero while its Legacy accuracy falls to 29.9% and grows as accuracy recovers to 97.8%; post-Adam's never forms, and its answers do not return. Beyond Mistral-7B. The same signature holds in all 28 paired contrasts on four more models and under full-parameter SFT.

We have also revised the manuscript acoordingly. Please check Section 4.2; Appendix N.1–N.2.

Together, these changes make the explanatory chain explicit: the decomposition identifies the term that carries the gap, the optimizer state explains why that term differs, and the moment-source intervention confirms it causally. We hope this resolves the main concern, and we would welcome any further questions.

Reviewer_rsrLOpenReview ↗

Response to Authors

Thank you for the detailed response and additional experiments. The new experiment on Adam’s first moment provides a clearer explanation of the retention gap between pre-Adam and post-Adam projection. The explanation of why the single-step experiments do not show the same gap is also helpful. These results address an important part of my original concern.

However, I still find the revised paper difficult to follow. The authors have added definitions, but the main text still uses many terms that readers need to remember and distinguish. For example, Sections 4.1–4.2 discuss Legacy reference-exact accuracy, independent strict-EOS accuracy, the full retention contract, and supporting New non-inferiority. It remains difficult to keep track of their different settings and understand which conclusion each result supports. Adding definitions and a glossary helps, but I think the main text still needs simpler explanations.

The organization has also improved with the research questions and separate subsections. However, the connection between the theoretical analysis, optimizer experiments, and generation results is still not sufficiently clear to me. The mechanism discussion in Section 4.2 contains many findings in one paragraph and refers to several appendix sections. It would help to explain this connection step by step, and clarify which conclusions follow from the analysis and which are supported by the additional experiments.

Overall, I acknowledge the improvements and the additional evidence. However, my concerns about the presentation and clarity of the explanation remain. I therefore maintain my original rating of 2 (weak rejection).

Thank you for the feedback. We have clarified the evaluation measures in Sections 4.1–4.2 and the connection between theory, optimizer interventions, and generation results, distinguishing theoretical conclusions from experimental findings. Please let us know if anything remains unclear; we welcome further questions.

Reviewer_gd5JOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates why gradient- and subspace-constrained updates can still cause catastrophic forgetting during continual supervised fine-tuning (SFT). The authors introduce a functional-path decomposition that attributes changes in old-task loss to four terms: the constrained update direction, probe mismatch, gradient staleness, and finite-step effects.

Through controlled QLoRA experiments, the paper examines constraint granularity, projection placement relative to Adam, replay timing, and intermediate retention trajectories. The results show that satisfying local constraints does not guarantee behavioral retention, and that similar final accuracies can conceal substantially different forgetting trajectories.

The problem is important, and the empirical investigation is thorough. However, the theoretical novelty is modest, and the generality of the empirical conclusions remains insufficiently established.

Critical strengths

1. Important problem and coherent analytical framework.: The paper addresses a meaningful gap between optimization-level constraints and functional retention. The decomposition in Proposition 1 provides a clear interpretation of why locally feasible updates may still increase old-task loss. The quadratic and cross-entropy counterexamples further demonstrate this limitation.

2. Carefully controlled empirical comparisons.: The factorial experiments disentangle constraint granularity from projection placement. The matched-seed design, numerical feasibility checks, and multiple-comparison corrections strengthen the credibility of the findings. In particular, the substantial retention difference between pre-Adam and post-Adam projection is noteworthy.

3. Valuable trajectory-level observations.: The checkpoint and replay-timing analyses demonstrate that endpoint accuracy alone is insufficient to characterize forgetting. For example, aggregate pre-Adam projection and replay achieve the same final legacy accuracy (97.83%) but exhibit markedly different intermediate retention. This motivates evaluating behavioral stability throughout adaptation rather than exclusively at convergence.

Critical weaknesses

1. Limited theoretical novelty and predictive value. Proposition 1 primarily follows from Taylor expansion and an algebraic decomposition of gradients. Corollary 1 relies on standard smoothness arguments, while Proposition 2 establishes a conventional conditional descent result. Although these results organize relevant factors, it remains unclear what new predictions they enable. In particular, the paper does not demonstrate that its theoretical bounds can quantitatively predict forgetting or guide practical decisions such as refresh-frequency selection.

2. Restricted experimental generality. Despite the large number of runs, the main experiments rely on relatively narrow tasks and evaluation sets. The original qualified generation banks contain only 23, 17, and 26 examples for Mistral, Qwen, and OLMo, respectively, and measure different behavioral categories across models. The central granularity–placement factorial is evaluated only on Mistral. Moreover, the sequential-learning comparison uses a single task order and two seeds. These limitations weaken claims about general continual SFT.

3. Insufficient mechanistic explanation of optimizer placement. Pre-Adam and post-Adam projection differ not only in where constraints are applied but also in how Adam's optimizer states evolve. Consequently, the observed retention differences identify the effects of two optimization procedures without isolating the underlying mechanism. Further analysis of update magnitudes, moment dynamics, and functional-path components would help explain why pre-Adam projection performs substantially better.

4. Replay effectiveness is strongly evaluation-dependent. Replay substantially improves retention on the code-label task but does not consistently preserve strict responses on other tasks. In Table 2, replay reduces independent strict-response accuracy by 8.96 and 22.71 percentage points on natural sentiment and structured extraction, respectively. Furthermore, replay satisfies the complete retention contract in only 1/8 runs in the main code comparison. These findings suggest that its benefits depend critically on the protected behavior and evaluation interface.

Questions for authors

Predictive utility: Can the functional-path decomposition provide a quantitative predictor of forgetting that outperforms simpler diagnostics, such as old-task NLL, gradient alignment, or update norms?

Optimizer mechanism: What explains the large advantage of pre-Adam projection? Does it persist when differences in optimizer states and update magnitudes are explicitly controlled or analyzed?

Cross-model generalization: Can the central granularity–placement results be reproduced on Qwen or OLMo, preferably using larger and more diverse old-task evaluation sets?

Replay behavior: What explains the deterioration in independent strict-response accuracy despite improvements on legacy evaluations? Can the authors distinguish insufficient probe coverage from changes in response formatting and stopping behavior?

2 comments

Thank you for a detailed and constructive review. Each point has a direct answer from new analyses or experiments. We use W for each weakness, Q for each question, and R for our response. Section and appendix numbers refer to the revised PDF; table numbers refer to the submission.

W1 / Q1. Limited theoretical novelty and predictive value: can the bounds predict forgetting or guide decisions such as refresh-frequency selection, and does the decomposition outperform simpler diagnostics?

R1. The value of the analysis lies in what it separates. Each separated term makes a testable prediction, and we have confirmed the most practical one, the choice of refresh interval, prospectively. Before any new run, we fixed a budget of 0.1 nats per run and two forecasts of the staleness cost of every refresh interval: a saturating one, read from the submitted refresh-8 runs alone, and a linear one that follows Corollary 1's path-length bound. We then ran 128 new runs: r = 1, 2, 4, …, 128, on Mistral-7B and Qwen2.5-7B, with eight seeds each. Staleness summed over a run, measured (saturating forecast):

Refresh interval rMistral-7B stalenessMistral-7B held-out forgetting vs r = 1Qwen2.5-7B stalenessRefresh share of training time
10 (0)—0 (0)32–33%
20.26 (0.36)+0.080.009 (0.020)19–20%
40.64 (1.06)+0.140.026 (0.041)11%
81.27 (1.94)+0.360.034 (0.038)6%
161.82 (2.38)+0.580.017 (0.029)3%
321.83 (2.59)+0.730.022 (0.024)1.5%
642.00 (2.70)+0.630.008 (0.022)0.7%
1281.48 (2.76)+0.870.005 (0.021)0.4–0.5%

The saturating forecast chose correctly on both models. It prescribed refresh at every step on Mistral, where r = 2 already exceeds the budget, and a single refresh per run on Qwen, where staleness stays below 0.035 nats at every interval. On Qwen this saves about a third of the training time with Legacy at 100% throughout; on Mistral, longer intervals add held-out forgetting. The ladder confirms the saturating shape, and the bound holds. Staleness saturates instead of growing with path length as the linear forecast assumed, and the per-update bound of Corollary 1 holds at 12,302 of 12,304 stale updates. The other propositions predict too. Proposition 1: the controlled term carries the entire placement gap (+1.47 of +1.42 nats), while mismatch and remainder stay flat. Proposition 2: pre-Adam follows the lower Old-loss path in all 28 paired contrasts across models. Simpler diagnostics (Q1). After 8 of 128 updates, the controlled term flags the forgetting runs best (AUROC 0.98, against 0.91–0.96 for old-task NLL; update norms and gradient alignment near chance). On the ladder, staleness tracks held-out forgetting (Pearson 0.73) far better than update norms (−0.12). Old-task NLL also tracks forgetting closely, since the four terms sum to it. The decomposition adds two things: attribution, which names the term that failed and hence the fix (placement, refresh or probe coverage); and forecasting, since the staleness profile of the submitted refresh-8 runs predicts the whole ladder.

We have also revised the manuscript acoordingly. Please check Appendix N.1, N.4 and N.5.

W2 / Q3. Generality: narrow tasks and evaluation sets, a factorial on Mistral only, and a sequential study with one task order and two seeds.

R2. Thanks. We added experiments and find that the placement result holds across five models ranging from 7B to 32B and under full-parameter SFT, while the sequential result remains consistent across all six task orders. Each added model is evaluated on two independently qualified 60-item Old banks with identical composition (arithmetic, literal copy, and instruction format), both disjoint from the probes and replay set:

Model (paired seeds)Old answers plain SFT keepsOld-loss path, post-Adam − pre-AdamOld answers kept at the end, pre-Adam / post-Adam
Mistral-7B (8, submitted)5.4%+0.40, 8/8 seeds97.8% / 7.1%
Qwen2.5-7B (4)100%+0.26, 4/4100% / 100%
Qwen2.5-14B (4)98.9%+0.28, 4/4100% / 97.8%
Mistral-Small-22B (4)*13.8%+0.39, 4/450.2% / 6.5%
Qwen2.5-32B (4)96%+0.20, 4/496% / 96%
Qwen2.5-1.5B, full-parameter (4)36.8%+0.81, 4/4100% / 93.4%

Endpoints use the aggregate constraint, as in Table 1. 22B is audited on its chat-format banks.

Placement generalizes. Post-Adam follows the higher Old-loss path in all 28 paired contrasts and ends with the larger Old-loss increase at each granularity separately (52 of 52). The endpoint gap appears wherever plain SFT forgets (bold). Granularity adds retention where placement leaves room on the accuracy scale, under both placements. This explains why the Table 1 interaction appears on Legacy, where pre-Adam is at ceiling, but not on the strict banks. The sequential result holds across all six task orders. With six seeds per order, replay keeps every earlier task in 35 of 36 chains, against 7–26 for the constraint-based methods.

We have also revised the manuscript acoordingly. Please check Sections 4.2 and 4.5; Appendix N.3 and N.6.

W3 / Q2. What explains pre-Adam's advantage once optimizer state and update magnitude are controlled?

R3. The advantage is carried by what Adam's first moment accumulates, and a causal test with the displacement constraint and step size held fixed confirms it. Not step size. Update norms are matched (0.1738 vs 0.1725), and the finite-step remainder, which carries the curvature effect of step size, does not change with placement. Optimizer state. Pre-Adam projection rewrites a conflicting gradient g as g + αh, so Adam's moments accumulate the probe gradient, the restoring form of Proposition 2. Post-Adam projection leaves the raw conflicting gradient in the moments and only trims each step, so its own candidate step stays Old-harming in every window and the projection should intervene at 74% of updates. Causal test. With post-Adam's exact displacement constraint kept at every update, feeding the projected gradient into the first moment raises Legacy retention from 6.5% to 94.6% (+87 to +91 points on every seed), with matched update norms and New accuracy. Feeding it into the second moment alone leaves 5.4%.

We have also revised the manuscript acoordingly. Please check Section 4.2 and Appendix N.2.

W4 / Q4. Replay's effectiveness depends on the evaluation: strict-response deficits on Natural and Extraction, and the 1/8 full-contract count.

R4. Replay's benefit is large wherever plain SFT destroys the audited behavior, and the strict-bank deficits come from answer form and coverage, not from stopping. Where it matters. Replay gains +70.31 strict-bank points over Plain on Code (positive in 8/8 seeds). In the sequential study, it keeps every earlier task in 35 of 36 chains, against at most 26 for any constraint-based method. Not stopping. No Extraction replay output reproduces the answer and then fails to stop. Answer form accounts for 17.50 of the 22.71 Extraction points and 5.00 of the 8.96 Natural points; on bank A, the typical miss is a leading space. Coverage explains the content errors: they fall on multiplication word problems, a form replay never supervises. This is the protected-object principle applied to replay. The 1/8 count answers a stricter question, whether every absolute Old and New budget holds at once; replay's improvement over Plain is positive in all eight seeds.

We have also revised the manuscript acoordingly. Please check Section 4.2 and Appendix N.7.

We hope these results address your concerns, and we are happy to answer any further questions.

Reviewer_xVf2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper explores why SFT leads to catastrophic forgetting, even when gradient projection or parameter constraints are employed. The authors propose a functional-path account, which decomposes the change in loss on previous tasks into several components, e.g., cached constraints, gradient staleness, probe mismatch, and finite-step residuals. Through both theoretical and empirical analysis, the study demonstrates that satisfying local constraints alone is insufficient to guarantee the preservation of model behavior. Extensive experiments show that supervised replay provides more stable preservation of behavior on previous tasks throughout the training process, whereas local constraint-based methods often necessitate trade-offs when attempting to retain performance across multiple prior tasks.

Critical strengths
  1. The paper makes a comprehensive theoretical contribution by introducing a functional-path decomposition. It clearly breaks down the components of loss change, cached constraints, probe mismatch, gradient staleness, and finite-step residuals—and elucidates why local feasibility is inadequate for ensuring the retention of model behavior.

  2. The experimental design is highly meticulous, effectively isolating confounding variables such as constraint granularity and the placement of constraints within the optimization process. Furthermore, the study distinguishes between intermediate loss trajectories and the final model behavior, revealing dynamic processes that are often overlooked when focusing solely on aggregate performance metrics.

  3. The research highlights that in sequential learning scenarios, different methods prioritize the retention of earlier tasks in distinct ways.

Critical weaknesses
  1. The primary large-scale experiments rely heavily on 7B-parameter models (such as Mistral, Qwen, and OLMo) and utilize QLoRA, all within a specific framework of instructions and formatting. Without further testing, it remains unclear whether the specific failure modes resulting from these constraints would persist in scenarios involving larger-scale models or full-parameter fine-tuning.

  2. In the sequential learning experiments evaluating backward transfer, the task chains are short—consisting of only three tasks and the evaluations are based on just two random seeds.

Questions for authors

see weakness

3 comments

Thank you for the positive assessment. We use W for each weakness and R for our response. Section and appendix numbers refer to the revised PDF.

W1. The main experiments use 7B models with QLoRA. It is unclear whether the failure modes persist for larger models or full-parameter fine-tuning.

R1. They persist at every scale we tested and under full-parameter fine-tuning, with the same mechanism. We ran the full structure–location factorial (constraint granularity × placement relative to Adam) on four more models (four paired seeds each, each audited on two newly qualified 60-item Old banks) and with every weight of Qwen2.5-1.5B trained (FP32 master weights):

Model (paired seeds)Old answers plain SFT keepsOld-loss path, post-Adam − pre-AdamOld answers kept at the end, pre-Adam / post-Adam
Mistral-7B (8, submitted)5.4%+0.40, 8/8 seeds97.8% / 7.1%
Qwen2.5-7B (4)100%+0.26, 4/4100% / 100%
Qwen2.5-14B (4)98.9%+0.28, 4/4100% / 97.8%
Mistral-Small-22B (4)*13.8%+0.39, 4/450.2% / 6.5%
Qwen2.5-32B (4)96%+0.20, 4/496% / 96%
Qwen2.5-1.5B, full-parameter (4)36.8%+0.81, 4/4100% / 93.4%

Endpoints use the aggregate constraint, as in Table 1. 22B is audited on its chat-format banks.

The mechanism appears on every model. In all 28 paired contrasts, post-Adam follows the higher Old-loss path and leaves a more Old-harming candidate step in Adam's state. The endpoint gap returns wherever plain SFT forgets (bold): Mistral-Small-22B and full-parameter SFT reproduce the Mistral-7B pattern. Where plain SFT keeps 96–100% of the answers (Qwen), placement shows in the loss path rather than at the endpoint: the path–endpoint distinction you highlight in Strength 2. The mechanism belongs to the optimizer, which is why it transfers. A causal test locates it in Adam's first moment: feeding the projected gradient into the first moment alone raises post-Adam's Legacy retention from 6.5% to 94.6%.

We have also revised the manuscript acoordingly. Please check Section 4.2; Appendix N.3.

W2. The sequential chains are short (three tasks), and the evaluation uses only two seeds.

R2. All six task orders run with six paired seeds each (36 paired chains, 216 runs), and the sequential result holds and sharpens. Each order reuses every method's submitted configuration and changes only the task order, the seed and the file paths. One adapter and its Adam state carry across the three tasks (32 updates each), and no method is re-tuned for a new order, so every task is twice the oldest, twice the middle and twice the newest. Over the 36 chains, keeping an earlier task means ending within 20 points of its just-learned accuracy:

MethodBWT change vs Plain [95% CI]Chains improvedChains keeping every earlier taskLargest earlier-task drop
Replay+25.8 [19.5, 32.1]34/3635/3621.9
Task-GEM+19.4 [12.9, 26.0]32/3626/3660.9
CLoRA+12.2 [5.2, 19.1]25/3615/3671.9
OPLoRA−1.1 [−7.6, 5.5]17/367/36100.0
Adapter EWC−1.6 [−5.5, 2.2]15/367/36100.0
Plain——6/36100.0

It holds in every order. Plain SFT forgets in every order (BWT −21.6 to −46.2), while replay's BWT stays between −3.1 and 0.0 and its gain over Plain is positive in every order. Two seeds were already enough for replay. Of the 630 possible pairs of chains, the size of the submitted study, 628 already show a positive mean gain for replay; CLoRA needs 16 chains before its mean gain is as reliable. Adding a task erodes subspace and regularization protection, not replay. The first two stages of every chain are themselves a two-task chain, so the study also contains all 36 two-task chains (every ordered task pair, six seeds). From two tasks to three, the chains keeping every earlier task go from 36 to 35 under replay, against 24 → 15 for CLoRA, 19 → 7 for OPLoRA and 18 → 7 for adapter EWC; Task-GEM, which constrains every step with stored examples of each earlier task, stays at 25–26. In the three-task chains, replay never loses the oldest task, the one that has passed through two later stages. Averages hide the tails. CLoRA's mean gain holds, yet it loses an earlier task in 21 of the 36 chains: the task-level trade-off you highlight in Strength 3.

We have also revised the manuscript acoordingly. Please check Section 4.5 and Table 3; Appendix N.6.

We hope these results address both concerns, and we are happy to answer any further questions.

Thanks for your detailed response. I will maintain my score.

Thank you for your recognition of our work!

Comments on the paper

1 comment

We thank all reviewers for their thoughtful feedback and recognition of our work. Following your suggestions, we have revised the manuscript and added further experiments to strengthen both the evidence and the presentation. If you feel that our rebuttal and revisions have addressed your concerns, we would greatly appreciate it if you could consider raising your scores. Thank you again for your time and constructive engagement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.