acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Transfer in Continual Learning: A Replay-Based Realisation

Abstract

Continual learning studies how deployed language models can continually acquire new tasks without expensive retraining from scratch. Existing methods have largely focused on preventing forgetting, leaving a complementary question underexplored: when can past experience actually help a new task? We identify three measurable conditions for realised forward transfer: the target must have remaining headroom, transferred information must persist through continued optimisation, and replay must come from compatible prior tasks. These conditions motivate Transfer-Selective Replay (TSR), which uses a training-free gradient signature to route replay toward sources predicted to help the incoming task, while distillation maintains stability. Across heterogeneous and homogeneous continual-learning streams, model scales, and backbone families, TSR improves forward transfer and final performance without sacrificing retention. More broadly, our results suggest that continual learning should treat transfer alongside forgetting as a first-class design objective.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, transfer learning, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Tg1DOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper focuses on forward transfer in continual learning and provides a generic framework around it. The proposed framework provides conditions that guides whether forward transfer is helpful for improving new task performance and what helps the forward transfer. To identify the possibility of helpful forward transfer, the paper uses the concept of headroom which measures how much adding more new-task data (target data) helps to increase the target performance. To answer "what helps" the forward transfer, the paper identifies gradient agreement signal between previous task and new task. This motivates the authors to design the proposed transfer selective replay method. Empirical analysis are provided for evidence of the proposed conditions and comparison of the proposed method with relevant baselines across different evaluation benchmark and backbones.

Critical strengths

The proposed framework that provides the conditions for forward transfer followed by transfer aware replay is interesting. The empirical results are provided across benchmarks and backbones with standard deviations as well. Use of temperature to control task weight is interesting.

Critical weaknesses
  • Two confounding factors are very critical and are not sufficiently addressed:
    • First, the overfitting of new task data under low data budget. Please provide evidence of extent of overfitting on new task data. Adding more source data may act as a regularizer that prevents overfitting of new task and thus improve the generalization of new task. Table 1 does show that transfer depends on the source but does not rule out whether the improvement is because of actual knowledge transfer or reduced overfitting.
    • Second, the confounding effect of knowledge distillation (KD) is not studied clearly. Table 10 needs two additional baseline comparison: 1: no replay and no KD. 2: no replay and only KD. Table 10 already shows target improvement by KD when replay is already included which shows the signal that KD also helps toward forward transfer which disagrees with the statement made in the main paper: " distillation therefore acts primarily as an optimization anchor, rather than as the source of transferable knowledge". Please provide proper evidence in this regard.
  • The proposed conditions in section 4 are merely signals that support forward transfer and does not provide necessary condition for forward transfer. Proper theoretical analysis and reason that these conditions matter for forward transfer is not formally established. Further, selecting replay samples based on gradient alignment is not new, although may not focus on forward transfer specifically (examples listed below). This severely limits the novelty of the paper.
    • Riemer et. al., "Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference" - ICLR 2019
    • Aljundi et. al., "Online Continual Learning with Maximally Interfered Retrieval" - NeurIPS 2019
    • Abbes et. al., "Revisiting Replay and Gradient Alignment For Continual Pretraining of Large Language Models" - CoLLAs 2026
  • The paper has many clarity issues where the terminologies and notations are not properly defined before usage.
    • In abstract and introduction, the following terms are unclear and needs to be properly defined before using or authors should use simpler terminologies.
      • What do you mean by "actually help"? (line 14)
      • What do you mean by "remaining headroom"? (line 15)
      • What is "transferred information"? (line 16)
      • Please provide the citation for "on modern LLM benchmarks it is routinely reported as absent or negative". (line 41)
      • What is "target"? (line 52)
      • What is "transfer carrier"? (line 52)
      • What is "task signature"? (line 74)
      • "task signature routes each incoming task" - how? (line 74)
    • How do you compute transfer gain in Figure 2b?
    • jj is not defined in Equation 2.
    • Condition 2 in section 4.2 is too short and unclear.
      • This required multiple read to understand, please make it simpler and elaborate: "One-shot initialisation from a source-trained adapter yields an early advantage that continued target training washes out" (line 214)
      • What is L2 anchor?
      • What is mixing gain?
      • What is full gain?
      • "recovers the full gain" in line 240 is misleading because it is full by design and does not mean maximum possible transfer. Please clarify this statement.
    • In equation 4 and line 261, B is not defined before using. Please clarify why gradient is computed wrt only B matrix.
    • Line 268: please elaborate what is meant by "we add an equal-norm source update to a target update and measure the resulting change in held-out target loss".
    • Line 300: what dies it mean by "stored-signature alignment"? What is ρ\rho? ρ\rho is not defined before using.
    • In algorithm 1 and 2, α\alpha is not defined before using.
    • Line 323, τ\tau is not defined properly.
    • In Equation 5, ff is not defined. What is ff?
    • Line 371, what is "era-matched references"?
  • In Table 1, what about the accuracy instead of loss for classifier based tasks?
  • In section 4.3, it is stated that only gradient direction reliably identifies the best source. This is over-statement as only couple of metrics are tested and even the gradient direction is not very reliable (4/8). How much of forward transfer is lost when gradient is not used or when gradient picks the wrong source?
  • Why is performance of DER++ lower than ER?
  • The statement "Selection replaces curriculum" is an overstatement. (line 470)
Questions for authors

Please refer to the weakness section above.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/3)

We thank the reviewer for the thoughtful comments. We address each concern below.


W1. Low-data Overfitting Confound

We agree that the observed transfer gains may partly reflect reduced overfitting, and that regularization and knowledge transfer are not mutually exclusive. We conducted additional experiments to quantify overfitting and examine whether the benefits of compatible replay can be explained by generic regularization alone.

Overfitting under limited supervision. Across five classification targets, training on 50 target examples reaches nearly 100% training accuracy, while test accuracy ranges from 14.4% to 58.9%. Halving the training steps yields negligible improvements, whereas increasing the target budget to 500 examples improves test accuracy by 4.7–11.7 percentage points. These results confirm substantial overfitting in the low-data setting.

Controlling for generic regularization. We compare replay from highly compatible sources with replay from low-compatibility sources and shuffled versions of the selected sources, using matched training budgets. High-compatibility replay improves target test accuracy by an average of 2.06 percentage points over target-only training, compared with 0.06 points for shuffled replay and −0.08 points for low-compatibility replay. It outperforms shuffled replay on all five targets and low-compatibility replay on four of five, with average advantages of 2.00 and 2.14 points, respectively.

TargetTarget onlyHalf steps500 targetShuffled selectedLow compatibilityHigh compatibility
C-STANCE48.4%48.4%57.8%47.7%46.9%49.4%
FOMC53.7%53.8%61.9%53.2%54.1%54.5%
NumGLUE-cm47.0%47.0%58.7%48.2%47.5%51.0%
NumGLUE-ds14.4%14.6%19.1%14.0%14.8%16.5%
ScienceQA58.9%58.2%65.8%59.6%58.7%61.3%

Interpretation. These results suggest that the benefit of compatible replay cannot be fully explained by additional training data or generic regularization. Preserving meaningful source supervision and selecting compatible sources both matter for target generalization. Although source-dependent regularization may still contribute, the results support gradient compatibility as a useful signal for identifying beneficial historical sources.


W2. KD Confound

Please see Global Response II, Section 2 (Independent Contribution of Source Selection), where we report a controlled ablation comparing uniform and signature-weighted replay with and without KD. The results show that source selection provides additional gains under matched KD configurations.


W3. Three Conditions and Theoretical Novelty

We agree that our presentation may overstate the theoretical status of the three conditions. We do not claim that headroom, persistence, and compatibility form a universal set of necessary or sufficient conditions for forward transfer. Rather, they are empirically motivated factors that help characterize transfer in low-budget sequential LLM adaptation.

Specifically, headroom describes the opportunity for improvement under limited target supervision, persistence concerns whether benefits from historical information survive continued optimization, and compatibility characterizes which historical sources are beneficial. Our controlled experiments motivate these factors: target-data scaling reveals remaining learning opportunities; carrier comparisons show that initialization benefits diminish while replay sustains gains under our protocol; and pairwise transfer measurements demonstrate substantial variation across sources, including negative transfer.

We also acknowledge important limitations. Target-data headroom is not a strict upper bound on gains from complementary sources, replay is not the only possible mechanism for preserving prior knowledge, and gradient similarity does not guarantee beneficial transfer. We will therefore revise Eq. (2) and Sections 4.1–4.3 to present the three factors as empirical diagnostics and design principles, rather than a necessary and sufficient characterization of forward transfer.

For a detailed discussion of novelty and our distinctions from prior gradient-based methods, including MIR, MER, and Efficient MER, please see Global Response 1: Novelty and Relation to Prior Work.

AuthorsOpenReview ↗

Rebuttal by Authors (2/3)

W4. Clarity and Notation

We thank the reviewer for the detailed feedback on notation and clarity. We will consolidate the definitions, clarify the measurements, and revise ambiguous statements as detailed below.

(1) Terminology and wording

  • “Actually help” (l. 14): We will clarify this as improving performance on a new task beyond what its own training data provides.
  • “Remaining headroom” (l. 15): We will define this as the opportunity for further improvement when additional target-task data are available, as measured in Eq. (3).
  • “Transferred information” (l. 16): We refer to information carried from an earlier task, either through its data or through learned parameters.
  • Negative or absent forward transfer (l. 41): We will add references to Fisch et al. (2023), Zheng et al. (2024), and Mueller et al. (2024), which document source-dependent negative transfer in continual language learning.
  • “Source” and “target” (l. 52): The source is an earlier task, while the target is the task currently being learned. We will define both before their first use.
  • “Transfer carrier” (l. 52): The mechanism through which prior-task information remains available, such as replayed data or previously trained parameters.
  • “Task signature” (l. 74): A task-level representation computed from training data. Our implementation uses the mean LoRA-B gradient defined in Eq. (4).
  • “Routes each incoming task” (l. 74): Cosine similarities between the incoming task signature and historical signatures are converted into softmax sampling probabilities, which determine the historical sources used for replay.

(2) Experimental and algorithmic clarifications

  • Figure 2b, transfer gain: The difference in held-out target accuracy between training on a 50/50 source–target mixture and target-only training. Both use fresh adapters, matched target-label and optimization-step budgets, five classification targets, and five seeds.
  • Eq. (2), indices jj and tt: We will define jj as a historical source-task index and tt as the incoming target-task index before introducing Eq. (2).
  • §4.2, one-shot initialization: A source-trained adapter can initially benefit target learning, but the advantage diminishes during continued target training under our protocol.
  • L2 anchor: A persistent penalty μ∥θ−θsource∥2\mu\|\theta-\theta_{\mathrm{source}}\|^2 added to the target training objective.
  • Mixing gain: The accuracy improvement from source–target mixed training relative to target-only training.
  • Figure 3, “recovers the full gain”: Replay is normalized to 100% relative to the mixing reference by construction, not because it reaches a theoretical maximum. We will remove this wording and report absolute values and seed variability.
  • Eq. (4), LoRA-B gradient: With W′=W+BAW'=W+BA, randomly initialized AA, and zero-initialized BB, the initial gradient with respect to AA is zero, while the gradient with respect to BB provides a nonzero first-order learning direction.
  • Equal-norm intervention (l. 268): All gradients are evaluated at the shared initialization (θ0\theta_0). We compare held-out target losses after (θ0−ϵg^t)(\theta_0-\epsilon\hat{g}_t) and (θ0−ϵ(g^t+g^s))(\theta_0-\epsilon(\hat{g}_t+\hat{g}_s)), where g^t\hat{g}_t and g^s\hat{g}_s are normalized LoRA-B gradients. We will also define the source-only and additional-target references.
  • Stored-signature alignment (l. 300): Cosine similarity between precomputed task signatures, distinguished from similarity between sampled batch gradients. We will clarify the source-level aggregation used to compute Spearman correlations with observed loss reductions.
  • Algorithms 1–2, historical records: Each task contributes a record (gj,Mj,θj)(g_j,M_j,\theta_j), containing its signature, replay memory, and adapter snapshot. We will explicitly specify that the first task is trained without replay or KD.
  • Replay ratio (α\alpha): The probability of sampling a historical replay batch at each training step, set to α=0.5\alpha=0.5.
  • Routing weights and temperature (τ\tau): wj=softmax⁡j(cos⁡(gj,gt)/τ)w_j=\operatorname{softmax}_j(\cos(g_j,g_t)/\tau), with τ=0.1\tau=0.1. The weights determine the historical task sampling distribution.
  • Eq. (5), fθf_\theta and θref\theta_{\mathrm{ref}}: fθf_\theta denotes the model's next-token predictive distribution, and θref\theta_{\mathrm{ref}} is a frozen historical adapter. KD uses teacher-to-student KL over the shared vocabulary at temperature T=2T=2, averaged over non-padding prompt and answer tokens, with T2T^2 scaling.
  • Era-matched references (l. 371): Replay batches from task jj use the adapter snapshot θj\theta_j saved after that task. Current-task batches use the historical snapshot with the highest routing weight.
AuthorsOpenReview ↗

Rebuttal by Authors (3/3)

W5. Accuracy-Based Transfer Evaluation

We thank the reviewer for suggesting an accuracy-based evaluation of the transfer matrix. Table 1 uses held-out target loss because it captures changes in predictive confidence even when classification accuracy remains unchanged. We agree that accuracy provides an important complementary measure of downstream transfer. We therefore report the corresponding accuracy gains for all five classification targets below.

Accuracy gain (percentage points) relative to target-only training. Rows denote target tasks and columns denote source tasks.

Target ↓ / Source →C-STANCEFOMCMeetingBankPy150ScienceQANumGLUE-cmNumGLUE-ds20Minuten
C-STANCE—+1.0−1.4−0.6+0.5−0.2−1.0−1.5
FOMC+1.3—+0.4+0.1+0.5−0.2+0.8+0.1
ScienceQA+0.2+2.4+0.4+0.6—+1.0+0.8−0.2
NumGLUE-cm+0.7+1.2+1.6+2.3+0.5—+4.0+0.2
NumGLUE-ds−0.2+0.3+0.5+1.6+0.1+2.1—+0.4

Agreement between loss and accuracy. To assess whether the two metrics lead to similar source choices, we compare their best sources and compute Spearman correlations across source rankings for each target.

TargetLoss-best sourceAccuracy-best sourceSpearman
C-STANCEFOMCFOMC+0.94
FOMCC-STANCEC-STANCE+0.54
ScienceQAFOMCFOMC+0.96
NumGLUE-cmMeetingBankNumGLUE-ds+0.79
NumGLUE-dsNumGLUE-cmNumGLUE-cm+0.68

The loss-optimal source also maximizes accuracy for four of five targets, and the source rankings exhibit positive Spearman correlations ranging from 0.68 to 0.96. However, the metrics are not interchangeable: some sources reduce held-out loss without improving accuracy, and the optimal source differs for NumGLUE-ds.

These results indicate that held-out loss captures meaningful source-dependent transfer effects that are largely reflected in downstream accuracy, while also revealing differences that accuracy alone may not capture.


W6. Gradient Signature Reliability

Please see Global Response 4: Reliability of Gradient Signatures, where we quantify selection regret and compare signature-based source selection with oracle and random selection.


W7. DER++ Baseline Correctness

Please see Global Response 2: DER++ Baseline Correctness, where we report the corrected DER++ results following an implementation audit and hyperparameter sweep.


W8. Selection Replaces Curriculum

We agree that this statement was too strong. Curriculum design concerns the ordering of training experiences, whereas source selection determines how historical tasks contribute to subsequent learning. Our experiments do not establish that source selection eliminates task-order effects or replaces curriculum design. We will remove the statement “Selection replaces curriculum” and clarify that TSR complements curriculum learning by controlling the reuse of historical information.

Reviewer_VbeJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how previously learned tasks can improve new-task learning in continual learning. It identifies three conditions for forward transfer: headroom, persistence, and source compatibility. Based on these findings, it proposes Transfer-Selective Replay (TSR), which uses gradient-based task similarity to select beneficial replay data while employing distillation for stability. Experiments across multiple benchmarks and model families demonstrate improvements in forward transfer and retention.

Critical strengths
  1. The paper provides a different perspective on continual learning by focusing not only on preventing forgetting but also on how knowledge from previous tasks can help improve learning on new tasks.

  2. The proposed TSR method builds on the three identified conditions for forward transfer and uses gradient-based replay selection to improve new-task learning, while distillation helps preserve previous knowledge.

Critical weaknesses
  1. Although the paper presents replay from a forward-transfer perspective, the proposed method mainly combines existing techniques such as gradient-based task similarity, selective replay, and knowledge distillation. Similar ideas have been explored in prior work, so the methodological novelty of TSR itself appears somewhat limited.

  2. The paper identifies three important conditions for forward transfer: headroom, persistence, and source compatibility. However, only source compatibility is explicitly used to guide replay selection. Headroom and persistence are mainly studied through controlled experiments rather than incorporated into an adaptive training strategy.

  3. Although TSR improves overall performance, the gains over standard experience replay are relatively modest in several settings.

Questions for authors

Please refer to the weaknesses listed above.

1 comment
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their constructive feedback and for recognizing the potential value of our work. We address the concerns below.


W1. Novelty

Please see Global Response 1: Novelty and Relation to Prior Work, where we clarify our contribution relative to existing gradient-based selection and alignment methods, including LESS, MIR, and MER.


W2. Only compatibility is explicitly used in TSR

The three factors serve different roles in our framework rather than corresponding to three separate algorithmic modules. Headroom characterizes whether an incoming task has room to benefit from prior knowledge; persistence motivates retaining access to historical information through replay; and compatibility determines which historical sources to prioritize. TSR operationalizes persistence and compatibility, while headroom serves as a diagnostic condition for when forward transfer is possible. We will make this distinction explicit at the beginning of Section 5.


W3. Modest improvements over ER

We agree that the improvements over standard ER are moderate in absolute magnitude and will calibrate our claims accordingly. Under the same target data and replay-step budgets, TSR improves Overall accuracy over ER across all four settings by 1.0–2.8 percentage points, while also improving plasticity and backward transfer.

More importantly, our central question is whether compatibility-guided selection of historical replay sources provides an additional benefit beyond uniform replay and distillation. To isolate this contribution, we conducted a new ablation comparing signature-weighted and uniform replay under matched KD configurations (see Global Response 3: Independent Contribution of Source Selection). Signature-weighted replay improves immediate new task performance both without KD and with matched era-based KD, supporting an independent benefit from source selection.

Reviewer_yM2aOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

Instead of preventing forgetting, this paper asks when past tasks can help learn a new one. It argues that forward transfer needs (1) headroom on the target, (2) a carrier that persists during training (replayed data does, parameter initialisation does not), and (3) a compatible source. To pick the source, it uses the cosine between LoRA-B gradients at the shared initialisation. The resulting method, Transfer-Selective Replay (TSR), replays past data weighted by this similarity and adds a KL distillation term for stability. On TRACE-8 and NumGLUE-8 with 50 examples per task, TSR obtains the best Overall accuracy and the only positive BWT.

Critical strengths
  • Research question. Reframing continual learning from "how not to forget" to "how can the past help learn better" is timely and interesting. Modern pretrained LLMs seem to suffer less from forgetting than the literature assumes, so shifting attention to forward transfer is well motivated.
  • Method. The idea of using data selection, rather than parameter reuse, as the transfer carrier is simple and sensible. Section 5 is clearly written, and Algorithms 1 and 2 are easy to follow.
  • Figure 1. It positions the paper with respect to the literature and gives the reader a useful overview.
  • Evaluation. The paper reports negative results honestly. For example, it says that TSR is not best in Plas on NumGLUE, and that headroom vanishes on PP15 and at 7B.
Critical weaknesses

W1. Clarity of Section 4, which carries the main conceptual contribution.

  • Key details are left implicit and must be reconstructed by rereading. When is headroom measured: before training on task t, or after the preceding tasks? On which model: the base model or the current continually trained one? The same ambiguity affects S_t(N) in Eq. 3, "transfer gain" in Fig. 2, and the "gain recovered" in Fig. 3.
  • In Sec. 4.3 the gradient signature is introduced without saying with respect to which parameters and model. The reader learns only later that it is the LoRA-B gradient at the shared initialisation.
  • The reading of the plots is hard (e.g. Sec. 4.1, "Evidence"). Statements such as "equivalent to roughly doubling the supervision" and "recovering 35% of the headroom" are not traceable to the figure, and Fig. 3 is a bar chart with no values or error bars.
  • Several claims are stated with unjustified certainty. For example, "Continual replay therefore requires source selection rather than indiscriminate reuse" is drawn from a single 8x8 matrix on one benchmark. The same applies to "requires" in Eq. 2 and to "Selection replaces curriculum".

W2. Evidence for the gradient signature is thinner than the narrative suggests.

  • Top-1 source prediction is 4/8 (Table 2), and the "reliably identifies" wording is not supported by that number. The gap to GradB_norm is nil, and the other signatures are also low, so the evidence is weak for all of them.
  • The mechanism analysis (rho = .87 / .85) is based on only three classification targets.
  • The signature is computed at theta_0, but transfer happens while training on a model that has already moved. The paper does not test whether the signature remains predictive after several tasks.

W3. Memory and computational cost are not analysed, and may be large.

  • It is unclear how the gradient signatures g_j of past tasks are handled. If they are stored, they take memory. If they are recomputed, this needs past examples and extra compute.
  • Algorithm 2 keeps one snapshot theta_j per task as the distillation reference. If each snapshot is stored separately, as I assume, memory grows with the number of tasks. It could become comparable to, or exceed, the cost of storing all past examples and retraining from scratch, which is exactly what continual learning wants to avoid.
  • The authors should provide an analysis of storage, memory and runtime as a function of the number of tasks and examples, compared with the baselines.

W4. Novelty of the stability component. The distillation loss (Eq. 5) seems to be the standard LwF-style KL term, with the reference taken from past snapshots. If so, the paper should say it explicitly and position it as such, rather than as a new "stability mechanism".

W5. Baselines and experimental protocol.

  • DER++ numbers in Table 3 are suspiciously low compared with naive ER, even though DER++ is ER plus logit replay and is usually at least as strong. A tuning issue, or a difference in buffer size or setting, should be ruled out.
  • ER is reported at two buffer sizes (10% and 50%), but the other rehearsal methods (A-GEM, DER++) are not. This makes the comparison hard to interpret.
  • It is not shown how much of the gain comes from selection, and how much comes from the combination of replay and distillation. A clean control would be uniform replay with the same distillation term and the same snapshots, at identical buffer and replay budget. The ER+KD family (DER++) should be run under the TSR budget.
  • The protocol uses 50 examples per task and is called the "standard sequential protocol". To my knowledge, TRACE and the cited works use much larger per-task training sets. So the numbers are not comparable with the literature, and all baselines are re-run in a regime that may favour replay.
  • Several differences in Table 3 are within one standard deviation, and some bolds do not coincide with the best value (e.g. NumGLUE 7B Plas, where SeqFT is higher than TSR). Claims such as "consistently improves" need significance tests or at least a more cautious wording.

W6. Presentation of the experiments.

  • Section 6 is heavily compressed. Almost all supporting tables (per-task results, backbones, ablations on routing and order) are in the appendix. The reader cannot form an informed view of the real experimental contribution from the main text.
  • The discussion of results is very dense and hard to follow.
  • The reproducibility statement mentions code only for the core TRACE-8 experiments.

W7. Novelty with respect to gradient-based selection and replay-selection methods is not clearly established.

  • Using gradients as the criterion to select replay data is not new. GSS (Aljundi et al., 2019b) and MIR (Aljundi et al., 2019a) select replay samples by gradient diversity or interference. GEM/A-GEM and MER (Riemer et al., 2019) use inner products and alignment between gradients of different tasks. LESS (Xia et al., 2024) selects training data for a target using the cosine between LoRA gradients, which is very close to the proposed signature. TAG (Fifty et al., 2021) uses gradients to estimate transfer between tasks.
  • The paper does cite these works, but mostly in a list, and it does not discuss precisely what differs from them. I acknowledge the genuine difference in objective (forward transfer rather than retention, and task-level routing at training time rather than sample-level buffer management). Still, the technical novelty is the combination of a training-free signature at the shared initialisation, task-level routing and era-matched distillation, and the paper should state this explicitly instead of presenting the gradient signal itself as the contribution.
  • More importantly, the experiments contain no replay-selection baselines. The compared methods are SeqFT, regularisation/isolation methods, and generic replay (A-GEM, DER++, ER). Without MIR, GSS or a LESS-style selection at equal budget, it is impossible to tell whether the gain comes from the proposed gradient signature or from any reasonable selection criterion.
Questions for authors
  1. Is the main hypothesis that the inner product (cosine) between the gradients of two tasks, computed with respect to the base model (theta_0), is predictive of transfer from one task to the other? Why is the base model the right reference point when training occurs on the continually updated adapter? Does the signature remain predictive at later stages of the stream?
  2. How are the signatures g_j and the snapshots theta_j stored? What is the total memory (in parameters and examples) after K tasks, relative to storing the replay buffers and the retrain-from-scratch alternative?
  3. Is the distillation loss identical to LwF, apart from the choice of the reference model?
  4. What are the DER++ hyperparameters and buffer sizes? Why are only ER results reported at two buffer sizes?
  5. Can you report a control with uniform replay plus the same distillation, and the standard-size TRACE protocol (larger per-task data)?
  6. When exactly is headroom measured (before task t or after task t-1, on the base model or the current one)? Is it predictive of transfer gain for a given task quantitatively, not just in aggregate trends?
  7. How does TSR compare, at equal budget, with replay-selection baselines such as MIR, GSS and a LESS-style gradient-based selection? What is the technical difference from LESS and from gradient-alignment methods such as GEM and MER, beyond the setting?
3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/3)

We thank the reviewer for the detailed feedback. We address the concerns below and will incorporate the additional analyses and clarifications into the revised manuscript.


W1. Clarity of Section 4

(1) Headroom and transfer measurement. All analyses in Section 4, including St(N)S_t(N) in Eq. (3), transfer gains in Fig. 2, and the persistence comparisons in Fig. 3, use independent fresh-adapter experiments rather than measurements along the continual-learning stream. Each experiment starts from the same shared adapter initialization θ0\theta_0, with the pretrained backbone frozen. Target-only and source-assisted configurations use this common initialization. We will state this protocol at the beginning of Section 4 and clarify the figure captions.

(2) Gradient signature definition. The signature in Eq. (4) is computed at the shared initialization θ0\theta_0, using gradients with respect to the LoRA-B matrices across all adapted layers. The pretrained backbone remains frozen. We will specify both the evaluation point and parameter subset when introducing the signature.

(3) Quantitative claims. We agree that the numerical claims accompanying Figs. 2–3 should be directly verifiable. At N=10N=10, target-only accuracy is 0.397, increasing to 0.463 with the selected source. This nearly matches the target-only accuracy of 0.464 at N=20N=20, motivating the interpretation that source-assisted learning provides a gain comparable to doubling target supervision. Relative to the target-only reference at N=500N=500, the available headroom is (0.588−0.397=0.191)(0.588-0.397=0.191). The source-assisted gain of (0.066)(0.066) therefore recovers 34.6% of this measured headroom. We will report these values and calculations alongside the corresponding figures.

(4) Scope and certainty of claims. We agree that several statements in Section 4 were too strong and will revise them as follows:

  • Source selection: The heterogeneous transfer effects observed on TRACE-8 motivate compatibility-aware replay, but do not establish its universal necessity. We will replace “continual replay therefore requires source selection” with “these results motivate compatibility-aware replay selection in heterogeneous task streams, where indiscriminate source reuse can introduce negative transfer.”
  • Eq. (2): Headroom, persistence, and compatibility are empirically motivated diagnostic factors, not formally established necessary conditions. We will remove the logical necessity formulation.
  • Selection and curriculum: TSR selects historical replay sources under a fixed incoming task order; it does not replace curriculum design. We will describe source selection as a complementary mechanism for controlling historical knowledge reuse.
AuthorsOpenReview ↗

Rebuttal by Authors (2/3)

W2. Evidence for Gradient Signatures

(1) Source-selection reliability. Please see Global Response 4: Reliability of Gradient Signatures for our analysis of oracle agreement, selection regret, and comparison with random selection.

(2) Extended mechanistic analysis. We extended the analysis to all eight TRACE tasks. For each target, we average intervention outcomes for each candidate source before computing Spearman correlations across sources. Across three perturbation scales, gradient alignment remains positively associated with local loss reduction:

ComparisonSpearman ρ\rho range
Batch-gradient cosine vs. source-only loss reduction0.92–0.98
Batch-gradient cosine vs. marginal loss reduction0.81–0.92
Stored-signature cosine vs. marginal loss reduction0.85–0.91

The ordering also persists across perturbation scales. Sources with higher alignment consistently yield larger marginal reductions in held-out target loss:

Source alignment group0.25×0.5×1.0×
Top third−0.115−0.199−0.241
Middle third−0.055−0.099−0.128
Bottom third−0.029−0.039−0.028

(3) Predictive utility of shared-initialization signatures. We compute task signatures at a common initialization to obtain a stable reference for comparing historical and incoming tasks, rather than estimating their instantaneous gradient alignment at each training stage. To evaluate this choice, we compare the predictive correlations and realized transfer gains of initial and current-checkpoint signatures.

MetricTasks 3–4Tasks 5–8
Initial cosine vs. realized transfer, Spearman ρ\rho+.70+.40
Current cosine vs. realized transfer, Spearman ρ\rho+.40+.10
Initial vs. current source rankings, Spearman ρ\rho+.45+.10

Initial-signature similarity remains more predictive of realized transfer than current-checkpoint similarity at both stages, although both weaken later in the stream.

Source-selection strategyTasks 3–4Tasks 5–8
Oracle-best source+.120+.105
Initial signature+.105+.082
Current signature+.095+.064
Uniform random source+.060+.045

Consistent with the correlations, initial-signature selection yields larger mean held-out loss reductions than both current-checkpoint selection and random selection at early and late stages. At later stages, it retains approximately 78% of the oracle improvement (0.082 versus 0.105), despite declining predictive correlation. Together, these findings support shared-initialization signatures as a useful and reusable reference for task compatibility.


W3. Memory and Computational Cost

Overall TSR requires additional storage relative to ER, but its measured footprint remains modest at the evaluated scale with little runtime overhead beyond matched replay with KD.

Storage. TSR retains one gradient signature and one LoRA adapter snapshot per historical task, while sharing a frozen backbone. At 0.5B, each task requires approximately 3.0 MB for its FP32 signature, 4.1 MB for its BF16 adapter snapshot, and 0.07 MB for replay examples. After eight tasks, this amounts to 57.6 MB, compared with approximately 0.5 MB for ER. The corresponding TSR footprint at 7B is approximately 267 MB. Although storage scales linearly with the number of tasks, it remains below 0.3 GB in our evaluated settings.

Runtime and GPU memory. On TRACE-8 at 0.5B, ER takes approximately 6.7 minutes, compared with 8.9 minutes for both TSR. Computing all task signatures requires approximately 8 seconds, and cosine-based routing takes less than 0.1 seconds. Peak GPU memory increases from 5.6 GB to 6.0 GB. Thus the additional runtime relative to ER is primarily attributable to distillation, while signature computation and routing introduce little overhead.


W4. Novelty of the Stability Component

We agree that the KL objective in Eq. (5) is standard knowledge distillation in the spirit of LwF, and we do not claim novelty in the loss formulation. The distinction lies in how historical teacher references are assigned within the replay procedure.

Original LwF distills predictions from a previous model on new-task inputs without requiring stored historical training data. TSR instead applies distillation to both current-task and replay batches, using stored LoRA adapter snapshots as teacher references. Distillation is performed over the language model's shared token vocabulary rather than separate task-specific output heads. The replay component is also related to DER/DER++, which preserve historical model predictions through stored logits. TSR instead retains task-era adapter snapshots that can generate teacher predictions for the corresponding batches. We regard this as a difference in reference assignment and implementation.

AuthorsOpenReview ↗

Rebuttal by Authors (3/3)

W5. Baselines and Experimental Protocols

(1) DER++ baseline. Please see Global Response 2: DER++ Baseline Correctness for our corrected results.

(2) ER with 10% replay. We agree that presenting both ratios without clarifying their different purposes was confusing. The additional 10% ER variant was included only to illustrate performance under a smaller replay budget, not as a matched comparison. We will remove the ER-10% row from the main table to keep the comparison focused.

(3) KD confound. Please see Global Response 3: Independent Contribution of Source Selection for our controlled comparisons of uniform and signature-weighted replay with and without KD.

(4) Experimental protocol. We follow the benchmark's sequential task ordering, single-adapter training, and evaluation procedure, while deliberately restricting target supervision to 50 labeled examples per task. This low-budget setting is motivated by our central research question since measuring forward transfer requires sufficient opportunity for improvement on the incoming task. We agree that describing this configuration simply as the “standard sequential protocol” is imprecise. We will instead use “low-budget adaptation of the standard sequential evaluation protocol”.

(5) Boldface and statistical uncertainty. We clarify that Table 3 correctly highlights SeqFT as achieving the highest plasticity on NumGLUE-8 at 7B. Regarding statistical reliability, we additionally examine paired comparisons using matched seeds, data splits, and task orders. On TRACE-8 at 0.5B, TSR improves Overall accuracy over ER by 2.4 percentage points, with a paired standard deviation of 0.8 points, a 95% confidence interval of [1.4, 3.4] points, and improvements in all five paired runs. We will include these paired statistics to better characterize the uncertainty of the reported gains.


W6. Experimental Presentation and Reproducibility

(1) Presentation of results. We agree that the main text should present the central evidence for TSR more directly. We will move the key routing ablations from Appendix B.4 and concise summaries of the per-task and backbone evaluations from Appendix C into the main experimental section, reducing reliance on supplementary material for interpreting the main findings.

(2) Interpretation and limitations. We will organize each experimental subsection around the comparison being tested, the observed results, and the conclusions they support. We will also discuss limitations more explicitly, including smaller source-selection gains on homogeneous tasks and reduced transfer headroom at larger model scales, rather than implying that the benefits are uniform across settings.

(3) Reproducibility. We will expand the documentation and experimental configurations to cover the additional evaluations, including their training settings and baseline implementations.


W7. Novelty and Replay-Selection Baselines

(1) Relation to prior work. Please see Global Response 1: Novelty and Relation to Prior Work for our comparison with gradient-based selection, replay, and alignment methods.

(2) Additional replay-selection baselines. To strengthen the empirical comparison, we evaluated MIR, LESS, and GSS-Greedy on TRACE-8 at 0.5B across five seeds, using a common 50% replay-step ratio.

MethodOverall ↑Plas ↑BWT ↑
MIR.445 ± .013.433+.012
LESS (5%).332 ± .026.427−.094
LESS (50%).435 ± .026.438−.003
GSS-Greedy (shared buffer).442 ± .015.437+.005
GSS-Greedy (100/task).446 ± .014.435+.012
TSR (ours).475 ± .030.439+.035

LESS. We evaluate both the original 5% selection ratio and a 50% variant to examine the effect of broader historical-data coverage.

GSS-Greedy. We consider the original shared-buffer setting, where examples from different tasks compete for a common memory budget, and a per-task variant retaining up to 100 examples per historical task, matching TSR's memory organization.

Results. TSR achieves the highest Overall accuracy and plasticity among the evaluated methods. These comparisons demonstrate the competitiveness of the complete TSR system against representative replay-selection methods. We will incorporate these additional baselines and their implementation details into the revised manuscript.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response (1/2)

We would like to thank all reviewers for their time and effort in evaluating our work and for their constructive comments and questions. We appreciate the detailed feedback, which has helped us clarify the scope of our contributions, strengthen the empirical comparisons, and improve the presentation of the paper.

We first address the main concerns shared across reviews, then respond to each reviewer's specific questions individually.


1. Novelty and Relation to Prior Work

We thank the reviewers for highlighting the connections to gradient-based selection and alignment methods. We agree that gradient similarity itself is not novel, and that prior work has already used gradient information to study transfer, interference, and data selection. We will revise our related-work discussion to position TSR more precisely. Table below summarizes the key methodological differences:

MethodSelection / intervention unitGradient-based signalReference pointPrimary objective
GSS (Aljundi et al., 2019b)Memory examplesGradient diversityCurrent modelRetention
MIR (Aljundi et al., 2019a)Replay examplesPredicted interferenceCurrent modelRetention
GEM / A-GEM (Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2019a)Parameter updatesGradient constraintsCurrent modelRetention
MER (Riemer et al., 2019)Parameter updatesGradient alignmentTraining trajectoryTransfer & retention
LESS (Xia et al., 2024)Training examplesGradient-based influenceWarm-up checkpointsTarget performance
TAG (Fifty et al., 2021)Task groupsCross-task gradient affinityJoint-training trajectoryMulti-task performance
Efficient MER (Abbes et al., 2026)Parameter updatesReptile-based gradient alignmentPretraining trajectoryRetention & efficiency
TSR (Ours)Historical replay tasksTask-gradient cosineFixed shared initializationTransfer & retention

The closest methodological precedent is LESS, which also uses LoRA gradients to identify beneficial training data. However, LESS estimates example-level influence using projected LoRA gradients across warm-up checkpoints for offline data selection, whereas TSR computes task-level signatures at a fixed shared initialization and uses them to weight historical replay sources during sequential learning. MIR retrieves replay examples, while GSS selects examples for memory retention, while MER and Efficient MER use gradient alignment to influence optimization rather than to rank historical replay tasks.

Scope of our contribution. We therefore do not claim novelty in gradient similarity, replay, or distillation individually. Our contribution is their specific integration for forward-transfer-oriented continual instruction learning: a training-free, shared-initialization task signature; compatibility-weighted historical replay; and task-era distillation references supporting retention. The distinctive decision is which past task to replay for an incoming task, rather than which individual examples to retain or how to modify the parameter update. We will make this distinction explicit in the revised manuscript and avoid claims of priority over gradient-based transfer methods generally.


2. DER++ baseline correctness

We thank the reviewers for identifying the anomalously low DER++ results in Table 3, which was due to unsuitable configuration. We re-evaluated DER++ with a sweep over the logit-matching weight and its published three-term formulation, using the same replay-buffer capacity as ER and TSR. The corrected results on TRACE-8 at 0.5B perform slightly better than ER:

MethodOverall ↑Plas ↑BWT ↑
ER.451 ± .031.427+.025
DER++, logit weight 0.1.456 ± .007.439+.018
DER++, logit weight 1.455 ± .003.436+.019
DER++, logit weight 10.445 ± .011.434+.011
DER++, logit weight 100.432 ± .011.432−.000
DER++, published three-term form.456 ± .024.437+.020
AuthorsOpenReview ↗

Global Response (2/2)

3.Independent Contribution of Source Selection

To disentangle replay selection from knowledge distillation (KD), we conduct a controlled ablation across two model scales, comparing no replay, uniform replay, and signature-weighted replay, with and without KD (Table below).

Source selection provides gains beyond KD. Without KD, signature-weighted replay improves immediate new task performance (Plas) over uniform ER by 0.8/0.6 percentage points on TRACE-8 (0.5B/7B). With era-matched KD applied to both replay strategies, the gains remain 0.9/0.4 points, respectively. This isolates a positive contribution from compatibility-guided replay selection beyond the distillation component.

KD primarily supports retention. Relative to sequential fine-tuning, KD-only improves BWT by 3.8/3.9 points, compared with just 0.6/0.5 points in Plas. When combined with replay, KD generally produces larger improvements in BWT than in Plas, consistent with its role in mitigating forgetting.

Combined performance. Signature-weighted replay with era-matched KD achieves the highest Overall accuracy at both scales, outperforming uniform replay with the same KD configuration by 1.2/1.8 points. Together, these results indicate complementary empirical contributions: source selection benefits new-task learning while KD mainly supports retention.

ReplayKD0.5B Overall0.5B Plas0.5B BWT7B Overall7B Plas7B BWT
NoneNone (SeqFT).264 ± .048.398 ± .015−.134 ± .058.584 ± .033.660 ± .001−.076 ± .031
NoneRecent KD.308 ± .042.404 ± .002−.096 ± .040.628 ± .027.665 ± .014−.037 ± .021
Uniform (ER)None.451 ± .031.427 ± .024+.025 ± .020.656 ± .034.662 ± .018−.006 ± .046
UniformEra-matched.463 ± .011.430 ± .010+.033 ± .020.666 ± .012.666 ± .010.000 ± .010
SignatureNone.460 ± .010.435 ± .017+.025 ± .016.670 ± .028.668 ± .013+.002 ± .032
Signature (TSR)Era-matched.475 ± .030.439 ± .017+.035 ± .015.684 ± .008.670 ± .012+.014 ± .011

4. Reliability of Gradient Signatures

We thank the reviewer for pointing out that 4/8 oracle-best source agreement does not support our original claim that gradient direction reliably identifies the best source. We agree and will revise this wording.

Beyond top-1 agreement. Top-1 accuracy does not capture how much transfer is lost when the selected source differs from the oracle. We therefore extend the analysis of the pairwise transfer matrix already reported in Table 1 by computing selection regret: the difference in held-out target-loss reduction between the hindsight-best source and the signature-selected source. Although signature selection identifies the oracle-best source on only 4/8 tasks (4/5 classification tasks), it achieves an average loss reduction of 0.464 nats, compared with 0.496 nats for the oracle, corresponding to 0.032 nats regret and 93.5% of the oracle improvement.

Comparison with random selection. Uniformly random source selection achieves only 0.368 nats of loss reduction, corresponding to 0.128 nats regret. Signature selection thus reduces regret by 75% relative to random selection, despite imperfect top-1 agreement.

These results support a narrower conclusion: gradient signatures provide a useful but imperfect source-compatibility signal, rather than a perfect oracle-best predictor. Importantly, TSR uses this signal for softmax-weighted replay, not hard source selection; its end-to-end benefit is evaluated separately in the matched replay/KD ablation above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.