acceptodds
Under review as a conference paper at ICLR 2027

Learning Dynamics of Continual Learning: A Unified View of Data Attribution, Forgetting, and Plasticity Loss

Abstract

Modern language models are likely to be updated throughout their lifetime rather than trained once and frozen. Each update therefore participates in a recurring cycle: decide which experience to learn from, understand what that update changes, and remain capable of learning from what comes next. We show that these challenges are governed by the same evolving update–behavior interaction. We derive a token- and layer-wise decomposition of how learning from one token changes another prediction. By separating the softmax force, shared readout geometry, and residual connections, it exposes two interaction channels and yields a forward-computable approximation. Following this interaction through time reveals a unified picture of continual adaptation. Positive interaction identifies useful experience; negative interaction produces either concentrated collision or accumulated erosion; over longer horizons, updates reshape the shared geometry mediating future learning signals, reducing their transmission. These predictions lead to effective data selection, mechanism-specific controls for interference, and a readout-based diagnostic of future learnability whose degradation predicts the benefit of restoring the readout. Across models and training regimes, the same local interaction thus explains both what an update changes now and how learning today changes what can be learned tomorrow. This view connects data attribution, forgetting, and plasticity loss as distinct regimes of the same evolving learning dynamics.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Continual Learning, LLM, Learning Dynamics, Data Attribution, Catastrophic Forgetting, Plasticity Loss

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zhjaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper connects data attribution, forgetting, and plasticity loss through first-order interactions between language-model updates. It separates a readout channel (CH1) from a backbone channel (CH2), using explicit approximations to obtain scores from forward quantities. Experiments examine attribution, cross-lingual retrieval, multilingual memory, and data selection. The authors distinguish interference from extreme updates ('collision') from accumulated weak interference ('erosion'). During prolonged PubMed training, a task-conditioned readout transmission score declines alongside downstream adaptability; restoring the readout partially recovers performance. The theory establishes local interactions and conditional directional attenuation, while the broader learning-trajectory account is supported empirically.

Critical strengths

The paper connects data selection, forgetting, and future learnability through a common update-behavior interaction. Its progression from local geometry to these three applications, supported by the overview and channel diagrams, helps readers follow the central motivation. This broad perspective also yields testable distinctions: on Qwen, adding CH2 improves cross-lingual MMLU retrieval from 0.488 to 0.625 when direct token overlap is limited. The combination of a coherent organizing framework and concrete predictions is useful for relating problems that are often evaluated separately.

The empirical study is substantial, combining controlled attribution, multilingual retrieval, data selection, interference interventions, and long-horizon adaptation across multiple models. These experiments test different aspects of the framework. This breadth makes the paper informative beyond a single benchmark result and gives readers several ways to assess the framework's practical utility and explanatory value.

Critical weaknesses
  1. The presentation does not consistently distinguish evidence from interpretation in the plasticity account. Section 5 uses 'formally prove' for the feedback loop, whereas Appendix A.3 describes the trajectory account as an extrapolation and Appendix B.3 proves attenuation under fixed forces and contraction. The empirical attribution also leaves three issues unresolved: Equation (7) varies with both readout geometry and task-force directions; the effect of retaining the final RMSNorm Jacobian is untested; and Figure 7(b) does not isolate whether the restoration benefit requires backbone updates. The scope of the plasticity argument is difficult to follow because its qualifications are distributed across Section 5 and Appendices A.3 and B.3. Stating the proof’s assumptions and limits alongside the main claim would help readers distinguish established results from the proposed feedback mechanism.

  2. The numerical results and comparative wording need separate corrections. Section 3 reports LESS as 0.587 for Qwen at 5%, whereas Table 5 gives 0.6171 versus 0.6356 for CH1+2: an advantage of 1.85, not approximately 4.9, percentage points. Separately, Table 1's broad superiority claim conflicts with HyperINF's higher MMLU retrieval score (0.694 versus 0.625). The first issue changes the reported gain; the second overstates where the method outperforms its competitors.

Questions for authors
  1. Please reconcile Section 5's 'formally prove' statement with Appendices A.3 and B.3. If the proof establishes conditional attenuation only, state that scope in the main text and present the feedback loop as a hypothesis. If a stronger result is intended, identify where the evolving feedback loop is established.

  2. On fixed held-out examples, compare R(Wt,g0)R(W_t,g_0), R(W0,gt)R(W_0,g_t), and R(Wt,gt)R(W_t,g_t) with baseline R(W0,g0)R(W_0,g_0), using Equation (7). Here 0 denotes the checkpoint before PubMed training. Does attenuation persist when probe forces are fixed? One task and a few saved checkpoints would distinguish contributions from readout changes and changing force directions.

  3. For the same task and checkpoints, evaluate E[∥JRMS(ht)⊤Wt⊤gt∥22∥gt∥22]\mathbb{E}\left[\frac{\lVert J_{\mathrm{RMS}}(h_t)^{\top}W_t^{\top}g_t\rVert_2^2}{\lVert g_t\rVert_2^2}\right], retaining the final RMSNorm Jacobian as specified in Appendix B.1. Does its change relative to the pre-PubMed baseline agree with the decline in raw RDR_D? This tests whether the diagnostic tracks transmission into the pre-normalization backbone.

  4. At one late PubMed checkpoint, compare reset versus no reset under both head-only and full-network adaptation, using matched data and update budgets. How much does the reset benefit differ between these two settings? Figure 7(b) already establishes the full-network benefit; the additional control would clarify whether that benefit requires backbone updates.

  5. For Qwen2.5-1.5B at the 5% selection budget, do Section 3's 0.587 and Table 5's 0.6171±0.00360.6171\pm0.0036 refer to the same configuration? Please identify the correct comparable LESS result and the corresponding CH1+2 gain. If the values come from different configurations, specify the difference and align the main-text comparison accordingly.

  6. Please revise Table 1's 'markedly outperforming gradient-based baselines' wording to reflect both retrieval columns. CH1+2 exceeds HyperINF on GSM8K (0.600 versus 0.370) but trails it on MMLU (0.625 versus 0.694); the caption should state the setting-specific advantage without implying superiority in both.

4 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for the careful and constructive assessment, particularly the insightful questions regarding the plasticity analysis. The main concerns center around the evidence supporting Section 5 and some imprecise claims about the experimental results in Section 3. We have revised these claims and restructured Section 5 to directly analyze the evolution of the readout geometry. We have also narrowed the scope of the "formally prove" statements and now present Section 5 as an empirical analysis with a discussion of possible mechanisms. The experiments suggested by the reviewer are also included in the appendix. Please refer to our overall response and the revised PDF for more details. Below, we respond point by point.

W1-Overall (same with Q1).

Thanks very much for raising these insightful comments on Section 5. We agree that the "formally prove" statement was too strong, and the evidence chain in the original Section 5 was not sufficiently supported: R_D depends on both readout geometry and task-force directions, and our theoretical results only hold for one or a few steps. Therefore, in the revised version, we have reorganized Section 5 to directly analyze the spectrum of w, rather than relying on the indirect scalar R_D, to better understand how the degeneration occurs. This is similar to what we did for the sequential SFT experiments in Appendix F, but with a more systematic analysis of the PubMed experiment. We believe that directly analyzing the spectrum allows us to make more precise claims and observations about how the readout degenerates.

We summarize this concern into the following four points and respond to them one by one.

W1-a (same as Q2). Equation (7) varies with both readout geometry and task-force directions…

Yes, that is correct. As also mentioned by reviewer JaP8, R_D requires samples from downstream tasks, which makes it less practical to use. Therefore, in the revised version, we directly analyze how the spectrum of w (and some submatrices of w) changes. This provides a more solid evidence chain, and the spectral diagnostics are also easier to compute in practice.

Regarding the R(Wt, g0) suggested in Q2, we believe the revised Section 5 no longer requires this analysis. However, since the observations are indeed interesting, we still provide them in Appendix G, Figure 29. In short, our results suggest that readout degeneration is more closely related to spectral redistribution than to an overall decrease in norm.

W1-b (same as Q3). The effect of retaining the final RMSNorm Jacobian is untested…

We agree that the RMSNorm Jacobian affects the gradient magnitude entering the backbone. However, even when accounting for this Jacobian, the gradients in CH2 are still mediated by the shared readout w. The readout geometry therefore remains a common component of the gradient propagation pathway, motivating our direct spectral analysis of wwT without modeling the additional Jacobian effects. In the revised Section 5, we no longer use R_D to predict plasticity loss, but instead focus on empirical spectral evolution and readout-reset interventions. We therefore believe that the proposed RMSNorm-specific experiment is no longer essential for the revised analysis, but we would be happy to include it in a subsequent revision if the reviewer considers it necessary.

W1-c (same with Q4). Figure 7(b) does not isolate whether the restoration benefit requires backbone updates…

Thanks very much for raising this concern. We agree that Figure 7(b) needs further ablations to understand where the recovery comes from. In addition to the experiment you suggested, we have conducted a full ablation under three settings. Specifically, after resetting w, we compare: (1) fine-tuning all parameters; (2) fine-tuning w only; and (3) fine-tuning the backbone only. Before resetting, we also try the three ablations above. The ablations show that resetting the readout improves both backbone-only and readout-only fine-tuning compared with their no-reset counterparts, while full-model fine-tuning achieves substantially lower losses than either restricted setting. This suggests that backbone updates remain important for recovering downstream plasticity. Please also refer to the new Figure7-c and Appendix G, Figure 28 for more details.

W1-d. Stating the proof’s assumptions and limits alongside the main claim …

We agree that the "formally prove" statement was too strong. The result in Appendix B.3 is conditional on specific assumptions and does not establish the full feedback loop. We have therefore removed this claim and now present the feedback loop as a possible explanation rather than a proven mechanism. The theoretical result in Appendix B.3 remains valid under its stated assumptions, and we have clarified its scope in the revised version.

AuthorsOpenReview ↗

Response (2/2)

W2 (same with Q5 and Q6). The numerical results and comparative wording need separate corrections.

We thank the reviewer for pointing out these issues, and we apologize for the numerical errors. The numbers in the main text were copied from an earlier version, while those in Table 5 are correct. We also agree that the caption of Table 1 was imprecise. We will correct these issues in the revised version. Please also refer to our overall response. We have also clarified that Table 5 used a different evaluation protocol, and we provide a new table in Appendix G, Table 16, comparing different evaluations.

Questions.

Please refer to our responses to the corresponding weaknesses above, where we discuss each of these questions.

Thank you for the detailed response. The LESS correction, the clarification of the GSM8K evaluation protocols, and the revised Table 1 caption address my concerns about the numerical comparisons. The revised Section 5 also gives a more defensible account of what Appendix B.3 establishes. In particular, the new ablations show that a readout reset can improve adaptation even when subsequent updates are restricted to the backbone.

My remaining concern is the link between this reset benefit and the proposed mechanism. Figure 29 shows a modest decline in readout transmission on out-of-distribution tasks with probe forces held fixed. How much this contributes to the loss of plasticity remains unclear. In particular, the spectral analysis and reset experiments leave open whether the learning signal entering the backbone weakens in the way described in the abstract, since the transmission analysis omits the final RMSNorm Jacobian. This link matters to the paper’s unified account of long-horizon learning, so I am keeping my overall rating at 2.

Thank you for the follow-up and for confirming that the numerical and evaluation concerns are resolved.

On the remaining point, we agree that our results do not quantify how much the measured transmission decline contributes to plasticity loss. We have revised the abstract and introduction accordingly: readout-mediated attenuation is now stated as a hypothesis consistent with the evidence rather than an established mechanism. More broadly, the revised Section 5 is intended as an empirical analysis of how the readout evolves under long-horizon training and how this relates to plasticity loss, rather than a formal account of the mechanism. We hope this clarified scope addresses your remaining concern.

Regarding the RMSNorm Jacobian: in the backbone-only ablation (Fig. 7c), the reset and no-reset runs start from the same backbone and the same final-norm parameters, so the normalization Jacobian is identical across the two conditions at the point of reset. The reset benefit under backbone-only updates therefore cannot come from the normalization layer and must be attributed to the change in the readout. What this intervention does not separate is whether the readout acts through its transmission geometry or through the forces it induces; we now state this limitation explicitly in Section 5. Because this comparison holds the normalization fixed by construction, we believe it isolates the readout’s contribution more directly than incorporating the Jacobian into the transmission diagnostic would.

Reviewer_JaP8OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies continual or lifelong learning in large language models. The core idea is to unify three different problem spaces (data attribution, forgetting and plasticity) under a common umbrella of token level update interactions on previously learned behavior (token level). The key measure being - how updating on a specific context token-pair affects the output predictions of a previously learned context-token pair. The authors factorize this interaction through three components - the softmax output and ground truth based interaction (termed token force), the shared read out matrix applied to the context embedding (last layer residual stream output), and the residual stream itself (through all the layers). Reasonable approximations with appropriate defense were used to simplify the final expression.

The final expression reveals two channels of interactions: CH1 - based on the token force (arises from the update related interactions between the differences in softmax output and ground truth weighted by the last layers representation similarity), and CH2 - a channel where the token forces are propagated through the read out layer, and then interact on the residual stream weighted by the hidden state similarities of all the layers. The key contribution of the paper is the derivation of the forward computable update interactions and studying the three problem spaces under various regimes identified using the decomposition of update interactions.

Data attribution is studied with respect to the sign and magnitude of the update interaction, where large and positive interaction based selection leads to better retrieval and downstream usage performance even against gradient based methods. Utilizing the decomposition, forgetting is divided into collision where a few token level updates cause forgetting (identified using the token energy, and avoided by not updating on those tokens), and erosion, which is a gradual accumulation of small negative interactions through commonly used response components such as the format (this is mitigated through separating the response formats for fine tuning and deployment). Finally, the authors also study plasticity loss, where long training causes the model difficulty in learning new tasks. The update interaction decomposition revealed one of the causes to be the reshaping of the output readout, resulting in new learning signals not being propagated to the residual stream and by extension the layers. The proposed resetting of the readout parameters as a diagnostic intervention resulted in a large portion of the performance to be regained as compared to starting with the base model on new tasks.

Critical strengths

The experiments conducted and the empirical results are compelling. The experiments are very well designed with each one specifically designed to answer a question and provide evidence. The evidence is also convincing. one such example is the case of dissociation between collision and erosion where the energy based masking has little to no impact on erosion showing the underlying cause is different. Another example is chain of evidence provided for the plasticity loss argument with declining Rayleigh quotient (RDR_D), slower adaptation / learning, readout resetting recovering the plasticity and the correlation between RDR_D decline and recovery.

Novel theoretical contribution by combining the token level factorization of the interactions, the channel decomposition and utilization of this to unify the three problems in continual learning.

Critical weaknesses

None

Questions for authors

Questions (does not affect my assesment) -

  1. The RDR_D computation requires knowledge of a new tasks samples which may not be available in deployment environment, can a mixture of unrelated task sample serve as an alternative?

  2. Can singular value analysis of the readout through training be a complimentary signal to the RDR_D, serving as task specific directions ?

1 comment
AuthorsOpenReview ↗

Thanks very much for liking our paper, we've re-oragnize Section 5 to make it easier to follow

We thank the reviewer for the encouraging assessment and for the two constructive questions. Both questions have motivated substantial improvements to our new Section 5! Please also refer to the overall response section and the new PDF. Here is the point-by-point response.

Q1. Can a mixture of unrelated task samples serve as an alternative …

We believe a mixture of tasks could provide a distribution-averaged proxy if it sufficiently represents the future tasks of interest. However, it would not necessarily capture the geometry relevant to an unknown downstream task. Our new Fig. 6(c) suggests this possibility, but through the geometry of wwT directly rather than RDR_{\mathcal D}.

To address this practical limitation, we have revised Section 5 to introduce spectral diagnostics that do not require downstream task samples or gradients. In particular, for long-horizon PubMed CPT, we examine the readout submatrices associated with the 1,000 most frequent PubMed tokens and their complement. This allows us to study distribution-dependent readout evolution using only information from the training distribution.

As shown in the revised Fig. 7(b), the PubMed-associated readout block remains relatively stable, while the complementary block exhibits substantial spectral concentration. This provides a downstream-task-agnostic diagnostic of geometric changes potentially relevant to future learnability, without claiming to predict plasticity for every unseen task.Having more information about the downstream task might be helpful. It is possible that knowing only the downstream token distribution would already provide useful information, as suggested by Fig. 6c, but we have not verified this in the CPT setting during rebuttal, sorry about that.

Q2. Can singular value analysis of the readout provide a complementary, task-specific signal?

Yes, exactly. We have substantially expanded this analysis in the revised Section 5 (Figs. 6 and 7). Specifically, we analyze the spectra of task-associated submatrices of ww⊤ww^\top using three complementary metrics: Trace (overall spectral strength), effective rank (effective dimensionality), and Density (spectral spread). All three are calculated from the eigenvalues.

In sequential SFT (Fig. 6c), these metrics generally increase for the task currently being trained, while those of other tasks decrease gradually or remain relatively stable. We further extend this analysis to long-horizon CPT (Fig. 7b), observing relative preservation of the training-associated readout geometry alongside stronger spectral concentration in the complementary block.

Importantly, these spectral changes accompany task-dependent plasticity loss (Fig. 7a), while restoring the readout recovers substantial downstream learnability (Fig. 7c). Together, these observations provide complementary geometric and functional evidence for the role of the shared readout in plasticity loss. Additional definitions, experiments, and interventions are provided in Appendix G.

Reviewer_5aJuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how one gradient update on a token u changes the log-probability of another token o. The authors apply a first-order expansion and separate the model into the softmax output signal, the shared readout matrix W, and the residual backbone. With an identity-path approximation of the residual stream, they obtain a forward-computable score (Eq. 5) with two channels: CH1 is the direct force alignment, and CH2 is the readout-mediated alignment, each scaled by hidden-state similarity.

The same score is then applied to three continual-learning problems. For data selection, candidates are ranked by their predicted benefit on a probe set; CH1 is enough when output tokens overlap, and CH2 helps in a cross-lingual setting. For forgetting, negative interactions are split into collision and erosion. Masking high-energy tokens reduces collision, and separating the fine-tuning format from the deployed format redirects erosion. For plasticity loss, they define readout transmission R_D as a Rayleigh quotient of WW^T along task forces. During long PubMed training, R_D decreases for OOD tasks, later checkpoints adapt more slowly, and resetting the readout to its base values partially recovers plasticity.

Experiments use Qwen2.5, Qwen3, Llama3.2, and OLMo models from 0.5B to 7B. The appendix notes that the signed score is unreliable at a cold start and needs about 100 SFT steps, and that the theory assumes SGD while experiments use AdamW.

Critical strengths

The main contribution is the separation of the interaction into two channels, CH1 and CH2. I see three concrete benefits.

S1. Two channels have very different computational cost, and the paper makes this visible. CH1 needs only one inner product over the vocabulary and one over the hidden state. CH2 needs a projection through the readout matrix and a sum over all layers. Earlier kernel-based views of learning dynamics put both into a single kernel, so it was not clear which part was expensive. With the separation, CH1 alone can be used when the output tokens overlap, at a cost similar to embedding similarity.

S2. The plasticity diagnostic falls out of the same formula. When the observed token and the updated token are the same, the CH2 term becomes the squared norm of the readout-projected force. Dividing it by the squared norm of the raw force gives the readout transmission score used in Section 5. So the diagnostic is not a new quantity added for that section. It is present in the decomposition from Section 2. This is the part of the unified view that I find real rather than rhetorical. It also follows from a simple fact about residual networks, namely that every layer receives the same output-side signal through the readout, which is why the readout appears as a bottleneck.

S3. CH2 is a sum over layers, so it can be cut at any depth. The authors use this to show where the sign error comes from at a cold start, by adding layers one by one from the output side. The same property reduces the cost when only some layers are trained or evaluated.

Critical weaknesses

W1. The main text does not match the tables. Section 3 reports LESS at 0.587 on Qwen2.5-1.5B with a 5 percent budget, but Table 5 reports 0.617. In the same table, random selection at 5 and 10 percent gives lower accuracy than the base model, which starts at 0.619. So fine-tuning on a small GSM8K subset hurts this model, and the gain of CH1 plus CH2 over the base model is about 1.6 points. At the 1 percent budget on Llama3.2-3B, both CH1 and CH1 plus CH2 are worse than random selection and LESS. The main text does not mention this. I ask the authors to correct the numbers and to include the base model in every comparison.

W2. The erosion experiment runs on a model whose target ability has already been lost. In Table 11, GSM8K accuracy before fine-tuning is 8.42 percent or zero. The same base model reaches 61.9 percent in Table 5. This means the stage that establishes the MMLU answer format has erased most of the GSM8K ability, and the paper never discusses this forgetting. The two directions are also not symmetric. Establishing Q/A and then training in P/R reaches 39 to 42 percent on GSM8K, while the reverse order reaches 57 to 58 percent. The statement that GSM8K is learned in all settings is true, but the levels are very different, and I do not think the two conditions can be compared as equals.

W3. The benefit of CH2 is reported selectively. The caption of Table 1 says that CH1 plus CH2 clearly beats gradient-based baselines. On cross-lingual MMLU, HyperINF reaches 0.694 and CH1 plus CH2 reaches 0.625. In Table 4, the main text quotes only Llama3.2-3B, where retrieval goes from 0.861 to 1.000. On that model, the embedding baseline and CH1 already reach 0.861. On Qwen2.5-1.5B and Qwen3-1.7B, adding CH2 lowers retrieval accuracy. In Table 5, at the 10 percent budget, CH1 alone is better than CH1 plus CH2 on both models. The paper should state clearly when CH2 helps and when it does not, instead of citing only the favorable cases.

Questions for authors

Q1. Table 1 marks CH1 and CH1 plus CH2 as not needing a backward pass. However, the signed score needs about 100 SFT steps of warm-up, and in Table 5 the score cost of CH1 plus CH2 on Llama3.2-3B is 3.12 GPU hours against 4.16 for LESS. How much of this cost is the warm-up, and how much is the readout projection over the vocabulary? I would like the two reported separately so that the forward-only claim can be judged fairly.

Q2. The proof in Appendix B.3 needs the readout to shrink a little at every step, and the text motivates this with weight decay. Figure 21 shows almost the same transmission curves with and without weight decay. What is the actual source of the shrinking when weight decay is off?

Q3. In practice, continual fine-tuning on small devices or private servers is often done with LoRA, and the readout is usually frozen. In that setting the CH1 term is zero and the readout cannot change, so the framework seems to predict no readout-driven plasticity loss. Have the authors tested this?

4 comments
AuthorsOpenReview ↗

Response (1/4)

We thank the reviewer for their effort in reviewing our paper. The main concerns center around the experimental details and how the results are summarized. We have revised the imprecise claims and clarified the experimental settings that might have caused misunderstandings throughout the paper. Please also refer to our overall response and the revised PDF. Below, we respond point by point.

Before addressing these concerns, we first apologize for an issue with the original Table 5. We believe several of the reviewer's concerns stem from this table.

The issue is that Table 5 used a non-standard GSM8K evaluator (results comparing different evaluators are provided in Appendix G). Specifically, we used a loose criterion that takes the first explicitly stated answer and ignores any numbers generated later in the response. This differs from the evaluation used elsewhere in the paper, which follows the standard zero-shot prompting and evaluation protocol (extracting the final answer at the end of the response).

The comparisons within Table 5 remain valid under this consistent evaluation criterion, but the results are not directly comparable with those in other tables. Furthermore, we find that base-version models can indeed achieve less than 10% accuracy on GSM8K before SFT with a simple zero-shot prompt, pass@1, and strict evaluation.

Low GSM8K accuracy for pretrained Base models has also been reported in prior work. For example, [1] reports only 1.3% initial Pass@1 for Llama-3.1-8B-Base. Authors of [2] also note that pretrained checkpoints do not reliably follow instruction formatting and might have low accuracy. Zhao et al. also report 6.8% GSM8K accuracy for Llama-3.2-1B-Base under 8-shot evaluation [3]. These observations are consistent with our finding that Base-model accuracy under zero-shot evaluation can be sensitive to the evaluation protocol.

For our purposes, these experiments aim to test whether our theoretically derived scores provide useful information for downstream data selection, rather than to establish state-of-the-art GSM8K performance.

[1] Heckel, Reinhard, Mahdi Soltanolkotabi, and Christos Thrampoulidis. "Asymmetric prompt weighting for reinforcement learning with verifiable rewards." ICML 2026

[2] Bansal, Rachit, et al. "RL Excursions during Pre-training: How early is too early for on-policy learning?" NeurIPS 2026

[3] Zhao, Changsheng, et al. "MobileLLM-R1: Exploring the limits of sub-billion language model reasoners with open training recipes." ICLR 2026

W1-a. Section 3 reports LESS at 0.587… Table 5 reports 0.617.

Sorry for this typo (0.587 was the result from an earlier version). We will correct the number in the text, and the one in Table 5 is correct.

W1-b. So fine-tuning on a small GSM8K subset hurts this model …

First, this observation only holds when we randomly select 5% and 10% GSM8K examples to fine-tune the Qwen model. For Qwen tuned with 1% data, Llama, and newly added OLMo, SFT improves performance compared with the model before fine-tuning.

We speculate that this is because the Qwen2.5 model is already very good at math problems, and some randomly selected examples might push the model away from its well-trained state. One piece of evidence is that Llama3.2-3B-base performs much worse than Qwen2.5-1.5B-base (despite being a larger model) on GSM8K, roughly 10% vs. 60%. Furthermore, across all selection methods, we find that adding more data hurts Qwen (performance decreases from 1% to 5% to 10%) but helps Llama and OLMo (as we would expect: more data means more information, hence better performance). As a result, we speculate that the relevant knowledge or capability might already be well encoded in the Qwen model.

We have also added an experiment on OLMo-1B to expand the scope of Table 5. The results are shown below:

Additional selection-then-SFT results on OLMo-1B.

Method1% (69)5% (348)10% (697)H100 hA100 h (est.)
Base (no SFT)0.0370.0370.037––
Random0.0830.1030.131––
LESS0.0850.1180.1210.5231.292
CH10.0980.1260.1480.0650.133
CH1+20.0910.1270.1470.2650.542

W1-c. I ask the authors to correct the numbers and to include the base model in every comparison.

Sure, we’ve corrected these typos and included those comparisons in the updated PDF. Together with the added experiments.

(To be continued...)

AuthorsOpenReview ↗

Response (2/4)

W2-a. The erosion experiment runs on a model whose target ability has already been lost.

First, we disagree with the claim that “The erosion experiment runs on a model whose target ability has already been lost.” This concern focuses only on the results in Table 11 in Appendix E.4 (related to lines 370–375 in Section 5) and Table 5 in Appendix D.3. It therefore only concerns the causal intervention experiment in Figure 5(e). Other results supporting the erosion claim, e.g., those in Figure 5(c–d), and all other results in Appendix E are not affected.

W2-b. In Table 11, GSM8K accuracy before fine-tuning is 8.42 percent or zero. The same base model reaches 61.9 percent in Table 5. …, and the paper never discusses this forgetting.

Second, we apologize for not clearly specifying the evaluation protocol used in Table 5. Both Table 5 and Table 11 use the Base versions of Qwen/Llama, but they adopt different GSM8K evaluation protocols. Table 5 uses a relaxed evaluator that extracts the explicitly stated numeric answer anywhere in the rollout, while Table 11 follows the stricter evaluation protocol, which extracts the final answer at the end of the response. This explains the large discrepancy in reported accuracy, rather than any forgetting caused by fine-tuning.

We agree that these numbers should not have been directly compared. We have clarified the evaluation settings in the revised PDF and report results under different evaluation protocols in Appendix G. Under standard evaluation, Base models can indeed achieve less than 10% accuracy on GSM8K before SFT (as stated at the beginning).

Regarding the choice of Base models in Table 11, we use them to construct a controlled 2×2 causal intervention. Since Instruct models already have preferences for particular response templates (Appendix E.4), starting from Base models allows us to establish these preferences explicitly before performing SFT under different templates.

W2-c. The two directions are also not symmetric.

We thank the reviewer for this observation. We acknowledge that different prompting formats can naturally lead to different fine-tuning dynamics and downstream performance. Importantly, our framework does not assume that the two prompting formats are symmetric, nor does the validity of the proposed erosion mechanism depend on such symmetry.

Our goal is not to demonstrate equivalent absolute performance across different prompting formats, but rather to investigate whether separating the fine-tuning and deployment formats can mitigate erosion-induced interference. Therefore, the relevant comparison is the improvement over the corresponding baseline under matched experimental conditions, rather than the absolute accuracy across different format setups.

In this experiment, we simply performed one epoch of SFT without tuning the training duration. We already observed a substantial difference in how well the original behavior was preserved, so we did not explore additional training epochs.

W2-d. The statement that GSM8K is learned in all settings is true, but the levels are very different, and I do not think the two conditions can be compared as equals.

Yes, the reviewer is correct that establishing a format preference before SFT may introduce additional forgetting. However, the purpose of this causal intervention experiment is to examine how SFT on GSM8K affects instruction-following ability on other tasks. We believe the first four columns of Table 11 already provide good evidence for this.

In short, this experiment is designed to show that template alignment indeed leads to more erosion. The difference between the third and fourth columns is large enough to support this observation. The other columns suggest that differences in training strength alone do not explain the observation, since the gap between the last two columns is much smaller than that between the third and fourth columns.

AuthorsOpenReview ↗

Response (3/4)

W3. The benefit of CH2 is reported selectively.

Sorry, but we think describing our results as "selectively reported" is too strong (and unfair) for our paper. Due to the page limits and the wide range of topics covered, we acknowledge that some of our wording and claims were too strong (also noted by reviewer zhja). However, we have reported all the raw numbers in the appendix for reference. For example, HyperINF indeed achieves the best performance on cross-lingual MMLU, but considering its computational complexity and performance on GSM8K, we summarized the results as "markedly outperforming gradient-based baselines." We agree that this wording should be more precise and will correct such issues in the revised version (see also our overall response). However, we do not think this constitutes selective reporting.

Regarding CH2's benefits, we have already stated that "For settings containing substantial task and output-token overlap, CH1 should already be highly informative." Thus, CH2 is expected to be more beneficial when direct token overlap is limited, as in the multilingual setting. This trend is clearly supported by Table 1, which follows standard comparisons in the data attribution literature. It is less consistent in the more complicated settings discussed in the appendix. For example, Table 4 considers a multilingual agent-memory retrieval task, where CH1+CH2 generally performs best but has some exceptions. Table 5 considers selection followed by SFT, where the benefits of CH2 are less consistent. One possible reason, as discussed in Appendix A.3, is that our decomposition only considers a single update step. Describing multi-epoch SFT would require an extended theory, which is beyond the scope of this paper.

In summary, we have added a short paragraph in Appendix G to discuss the limitations of CH2 in more detail. The general trend is that CH2 can provide additional information when token overlap is limited, but this advantage becomes less consistent after multiple epochs of training. In practice, since our scores are easy to compute, we can also track both scores periodically.

Q1-a Table 1 marks CH1 and CH1 plus CH2 as not needing a backward pass. ..

Thanks for this question. We apologize for not clearly distinguishing the settings that require warm-up from those that do not. In fact, warm-up is only used in Table 5, rather than being a general requirement of our method. We suspect that our discussion in Appendix C may have caused some confusion here. Appendix C investigates an interesting early-stage transition in token-level estimates, but this does not imply that warm-up is required for the data-sample-level benchmarks in Section 3.

For all tasks other than Table 5, we do NOT use warm-up: the scores are computed directly from the original model. Therefore, the claim in Table 1 that CH1 and CH1+CH2 do not require backpropagation is correct. We believe these data attribution benchmarks are designed to measure instantaneous similarity, and their evaluation does not require training (e.g., retrieving a paraphrase of sample x from other candidates). We will clarify this in the revised version.

Q1-b. Table5’s score cost

For this experiment, we do use a warm-up phase, since we view this task as estimating average data attribution over the course of training, and warm-up may help the model adapt to the unfamiliar GSM8K distribution. We will provide more details on the computational cost in the revised Table5. In short, however, the warm-up cost is negligible compared with computing scores for all 7k GSM8K candidates. We only warm up for 25 training steps with a batch size of 4 (i.e., 100 examples, compared with 7k candidates).

Q2. The proof in Appendix B.3 ...

Thanks very much for pointing this out. We agree that the assumption used in the proof in Appendix B.3 contradicts the empirical results in Figure 21. This suggests that the decay of the norm of w might not be the main mechanism. Therefore, in the revised version, we have reorganized Section 5 to directly analyze the spectrum of w, rather than relying on the indirect scalar measure (R_{\mathcal D}), to better understand how the degeneration occurs. This is similar to what we did for the sequential SFT experiments in Appendix F, but with a more systematic analysis of the PubMed experiment. We believe that directly analyzing the spectrum allows us to make more precise claims and observations about how the readout degenerates. Please also refer to the revised Section 5 and our overall response.

In short, we agree that the assumption in Appendix B.3 is inconsistent with our empirical observations. We have therefore removed the claims of "formally proving" the plasticity-loss mechanism and now present it as a hypothetical explanation. The result in Appendix B.3 itself remains valid under its stated assumptions, for example, in classical RL agents where larger learning rates and stronger weight decay may be applied.

AuthorsOpenReview ↗

Response (4/4)

Q3. In practice, continual fine-tuning on small devices or private servers is often done with LoRA…

Thanks for raising this interesting question. We have not tested LoRA in this work (the paper already contains too much information), but we believe our analysis can be extended to LoRA-based fine-tuning. We briefly discuss the implications below.

What changes with LoRA? The main difference lies in how the parameter update θt+1−θt\theta_{t+1}-\theta_t in Eq. (2) is computed. With LoRA, each adapted weight matrix is updated through low-rank factors AℓBℓA_\ell B_\ell, so gradients are taken with respect to these factors rather than the original weights. The same chain-rule analysis can be applied, but the layer-wise terms need to account for the LoRA parameterization.

What remains? As the reviewer correctly noted, freezing the readout removes CH1 in Eq. (5). However, the outer readout-projected forces w⊤gw^\top g in CH2 remain unchanged in form, while the inner layer-wise terms need to be reformulated. The direct and diffused interactions (note that CH2 also has strong direct influence, because the diagonal of wwT is large) illustrated in Figure 2 may therefore remain relevant under LoRA.

More importantly, our analysis suggests that higher layers have a broader influence on gradient propagation, as their effects accumulate across the gradients of preceding layers. Intuitively, the gradient at a lower layer contains both a direct path and indirect paths passing through higher layers. Thus, even with a frozen readout, bottlenecks in upper layers may still suppress learning signals across many lower layers, potentially leading to similar plasticity loss. We leave a systematic investigation of this mechanism under LoRA to future work.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overall Response (1/2)

Since several concerns are shared across different reviews, we summarize the changes made in the revised PDF in this overall response. Revised parts of the PDF are highlighted in orange. We have also reorganized several experiments by moving tables, figures, and discussions from the appendix to the main text. These revisions are intended to clarify and better present our existing results, rather than to change the scope of the paper.

1. Revisions to Numerical Issues, Experimental Settings, and Imprecise Conclusions

  • Line 26, abstract, modify our third contribution to “and spectral analyses linking the evolution of shared readout geometry to plasticity loss and its partial recovery through readout restoration.” Since our new Section 5 no longer claims “formally show”.
  • Line 100, Section 1, modify third contribution to “For long-term adaptability, we analyze the evolving spectrum of the shared readout and observe task-dependent spectral reshaping alongside plasticity loss during long-horizon training. Restoring the readout partially recovers this lost plasticity, providing functional evidence that readout geometry contributes to future learnability.”
  • Slightly modify Figure1, highlight spectral analysis rather than R_D
  • Line 243, add “The benefit of CH2 varies across settings, as readout-mediated coupling may also introduce irrelevant token interactions; see Appendix G for further discussion.” In Appendix G, we also briefly discuss when adding CH2 is beneficial, providing some intuition behind these observations.
  • Line 252, fix the wrong number for LESS 0.587→0.617
  • Line 253, add the numbers before SFT “and 0.619/0.107 before SFT” (those are from original Table 5 under relaxed evalution.)
  • Line 256, add one sentence “Note that our method is not always the best compared with those baselines. The results only aim to show that the information provided by our scores can be transferred to training tasks.” We would like to emphasize that the purpose of these experiments is to validate the theoretical insights, rather than to claim universal superiority over existing methods. We hope our results will be assessed in this context.
  • Line 261, Caption of Table 1: We remove the claim “markedly outperforming gradient-based baselines” and replace it with “improving performance over CH1 alone.” Here, we do not intend to emphasize that our method outperforms other methods, but rather to show that adding CH2 is helpful when token overlap is limited, as demonstrated in Table 1.
  • Line 1735, Caption of Table 5: we add “The warm-up cost of our method is less than 0.0036 h for Qwen and 0.0019 h for Llama, both negligible. For LESS, the warm-up costs are approximately 0.12 h and 0.10 h, respectively. LESS trains for 4 epochs on 5% of the data (as suggested in the original paper), while we use only about 1.4% of the data for warm-up (100 arbitrarily selected examples). We now report these costs separately in the table. Readout projection is only required for CH2, so its cost is already included in the CH1+2 row.” We also report the GPU hours for warm-up and scoring separately.
  • Line 1743: Table 5. Add the performance of the “base” model and “Relaxed evaluation: any numeric answer”. The difference in evaluation is the key to making those numbers across different tables not directly comparable (only Table 5 is unique in terms of evaluation). We also report the results under different evaluation protocols in Appendix G.
  • Line 2029, Appendix E.4, for the P/R, Q/A experiment part. We add “We start from the base versions of these models because their instruction-tuned counterparts already have preferences for particular response templates. Starting from the base models allows us to construct a full 2×2 causal intervention to isolate the effects of different templates.” to explain why Table 11 uses the Base versions for this causal intervention.
  • Made the training settings more precise for different experiments (mostly specified in the captions of tables and figures)
    • Line 1628, caption of Table 2, specify we use Qwen2.5-1.5B-Instruct
    • Line 1740, first row of Table5, specify the models we used are Qwen/Llama-Base
    • Line 1947, Table 9 and Table 10, all instruct versions
AuthorsOpenReview ↗

Overall Response (2/2)

(Continue)

2. Reorganized Section 5 (Plasticity Loss)

Many results and discussions moved from the Appendix (and also suggested by the reviewers); please do not treat them as "rewriting the paper"; thanks.

We have reorganized Section 5 to establish a clearer evidence chain, replacing the original RDR_{\mathcal D}-based (which depends on both w and g) prediction and correlation analyses with direct spectral diagnostics and functional interventions.

First, we use sequential SFT (already in the appendix in the original submission) to illustrate plasticity loss and introduce a geometric explanation based on the shared readout ww⊤ww^\top in CH2. We then track task-specific spectral properties (Trace, eRank, and Density) to examine how the readout geometry evolves during training (Fig. 6).

Second, we extend the analysis to long-horizon CPT, showing that OOD plasticity loss is accompanied by spectral concentration outside the training-associated readout region, while the latter remains relatively stable (Fig. 7ab). Fig.7b is newly added, and Fig. 7a doesn’t change.

Finally, we demonstrate that restoring the readout substantially recovers plasticity (Fig. 7c), with additional readout-only and backbone-only ablations. Appendix G provides further results and discussion.

Together, these results provide complementary geometric and functional evidence for the readout as an important bottleneck in future learning, without claiming that spectral changes alone establish causality. We have also removed claims like a "formal proof" from this section, instead presenting our findings as empirical observations and mechanistic analyses, with conclusions calibrated to the strength of the evidence. The theoretical result in Appendix B.3 remains valid under its stated assumptions, but we no longer present it as a general explanation of plasticity loss.

3. Appendix G

  • Table 15: New results for data-selection then SFT, on OLMo-2-0425-1B. Same setting as Table 5. But GPU hours are reported in H100 hours; they cannot be directly compared with those numbers in Table 5 (reported in A100 hours)
  • Table 16: original Table 5, under different evaluation protocols
  • A paragraph discussing when CH2 is beneficial, from an empirical perspective
  • Extra experiments for the revised Section 5
    • Figures 26, 27: More about spectral diagnostics
    • Figure 28: more on ablation studies on SFT readout or backbone only
    • Figure 29: R(Wt, g0)

4. Others

  • With the page limit increased from 9 to 10 pages, we have moved Table 6 from Appendix E to Section 4. The table summarizes the form of the update force gug_u under different training objectives relevant to continual learning and forgetting. We find that this energy-based perspective also provides useful insights into what these methods can mitigate and what they may fail to mitigate. We have also added a short paragraph discussing why erosion is harder to detect and why it may be particularly important in safety-related scenarios. The proposed analysis framework provides a useful tool for understanding and tracking these influences, which might be interesting for future exploration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.