- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper focuses on forward transfer in continual learning and provides a generic framework around it. The proposed framework provides conditions that guides whether forward transfer is helpful for improving new task performance and what helps the forward transfer. To identify the possibility of helpful forward transfer, the paper uses the concept of headroom which measures how much adding more new-task data (target data) helps to increase the target performance. To answer "what helps" the forward transfer, the paper identifies gradient agreement signal between previous task and new task. This motivates the authors to design the proposed transfer selective replay method. Empirical analysis are provided for evidence of the proposed conditions and comparison of the proposed method with relevant baselines across different evaluation benchmark and backbones.
Critical strengths
The proposed framework that provides the conditions for forward transfer followed by transfer aware replay is interesting. The empirical results are provided across benchmarks and backbones with standard deviations as well. Use of temperature to control task weight is interesting.
Critical weaknesses
- Two confounding factors are very critical and are not sufficiently addressed:
- First, the overfitting of new task data under low data budget. Please provide evidence of extent of overfitting on new task data. Adding more source data may act as a regularizer that prevents overfitting of new task and thus improve the generalization of new task. Table 1 does show that transfer depends on the source but does not rule out whether the improvement is because of actual knowledge transfer or reduced overfitting.
- Second, the confounding effect of knowledge distillation (KD) is not studied clearly. Table 10 needs two additional baseline comparison: 1:
no replay and no KD. 2:no replay and only KD. Table 10 already shows target improvement by KD when replay is already included which shows the signal that KD also helps toward forward transfer which disagrees with the statement made in the main paper: " distillation therefore acts primarily as an optimization anchor, rather than as the source of transferable knowledge". Please provide proper evidence in this regard.
- The proposed conditions in section 4 are merely signals that support forward transfer and does not provide necessary condition for forward transfer. Proper theoretical analysis and reason that these conditions matter for forward transfer is not formally established. Further, selecting replay samples based on gradient alignment is not new, although may not focus on forward transfer specifically (examples listed below). This severely limits the novelty of the paper.
- Riemer et. al., "Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference" - ICLR 2019
- Aljundi et. al., "Online Continual Learning with Maximally Interfered Retrieval" - NeurIPS 2019
- Abbes et. al., "Revisiting Replay and Gradient Alignment For Continual Pretraining of Large Language Models" - CoLLAs 2026
- The paper has many clarity issues where the terminologies and notations are not properly defined before usage.
- In abstract and introduction, the following terms are unclear and needs to be properly defined before using or authors should use simpler terminologies.
- What do you mean by "actually help"? (line 14)
- What do you mean by "remaining headroom"? (line 15)
- What is "transferred information"? (line 16)
- Please provide the citation for "on modern LLM benchmarks it is routinely reported as absent or negative". (line 41)
- What is "target"? (line 52)
- What is "transfer carrier"? (line 52)
- What is "task signature"? (line 74)
- "task signature routes each incoming task" - how? (line 74)
- How do you compute transfer gain in Figure 2b?
- is not defined in Equation 2.
- Condition 2 in section 4.2 is too short and unclear.
- This required multiple read to understand, please make it simpler and elaborate: "One-shot initialisation from a source-trained adapter yields an early advantage that continued target training washes out" (line 214)
- What is L2 anchor?
- What is mixing gain?
- What is full gain?
- "recovers the full gain" in line 240 is misleading because it is full by design and does not mean maximum possible transfer. Please clarify this statement.
- In equation 4 and line 261, B is not defined before using. Please clarify why gradient is computed wrt only B matrix.
- Line 268: please elaborate what is meant by "we add an equal-norm source update to a target update and measure the resulting change in held-out target loss".
- Line 300: what dies it mean by "stored-signature alignment"? What is ? is not defined before using.
- In algorithm 1 and 2, is not defined before using.
- Line 323, is not defined properly.
- In Equation 5, is not defined. What is ?
- Line 371, what is "era-matched references"?
- In abstract and introduction, the following terms are unclear and needs to be properly defined before using or authors should use simpler terminologies.
- In Table 1, what about the accuracy instead of loss for classifier based tasks?
- In section 4.3, it is stated that only gradient direction reliably identifies the best source. This is over-statement as only couple of metrics are tested and even the gradient direction is not very reliable (4/8). How much of forward transfer is lost when gradient is not used or when gradient picks the wrong source?
- Why is performance of DER++ lower than ER?
- The statement "Selection replaces curriculum" is an overstatement. (line 470)
Questions for authors
Please refer to the weakness section above.
3 comments
Rebuttal by Authors (2/3)
W4. Clarity and Notation
We thank the reviewer for the detailed feedback on notation and clarity. We will consolidate the definitions, clarify the measurements, and revise ambiguous statements as detailed below.
(1) Terminology and wording
- “Actually help” (l. 14): We will clarify this as improving performance on a new task beyond what its own training data provides.
- “Remaining headroom” (l. 15): We will define this as the opportunity for further improvement when additional target-task data are available, as measured in Eq. (3).
- “Transferred information” (l. 16): We refer to information carried from an earlier task, either through its data or through learned parameters.
- Negative or absent forward transfer (l. 41): We will add references to Fisch et al. (2023), Zheng et al. (2024), and Mueller et al. (2024), which document source-dependent negative transfer in continual language learning.
- “Source” and “target” (l. 52): The source is an earlier task, while the target is the task currently being learned. We will define both before their first use.
- “Transfer carrier” (l. 52): The mechanism through which prior-task information remains available, such as replayed data or previously trained parameters.
- “Task signature” (l. 74): A task-level representation computed from training data. Our implementation uses the mean LoRA-B gradient defined in Eq. (4).
- “Routes each incoming task” (l. 74): Cosine similarities between the incoming task signature and historical signatures are converted into softmax sampling probabilities, which determine the historical sources used for replay.
(2) Experimental and algorithmic clarifications
- Figure 2b, transfer gain: The difference in held-out target accuracy between training on a 50/50 source–target mixture and target-only training. Both use fresh adapters, matched target-label and optimization-step budgets, five classification targets, and five seeds.
- Eq. (2), indices and : We will define as a historical source-task index and as the incoming target-task index before introducing Eq. (2).
- §4.2, one-shot initialization: A source-trained adapter can initially benefit target learning, but the advantage diminishes during continued target training under our protocol.
- L2 anchor: A persistent penalty added to the target training objective.
- Mixing gain: The accuracy improvement from source–target mixed training relative to target-only training.
- Figure 3, “recovers the full gain”: Replay is normalized to 100% relative to the mixing reference by construction, not because it reaches a theoretical maximum. We will remove this wording and report absolute values and seed variability.
- Eq. (4), LoRA-B gradient: With , randomly initialized , and zero-initialized , the initial gradient with respect to is zero, while the gradient with respect to provides a nonzero first-order learning direction.
- Equal-norm intervention (l. 268): All gradients are evaluated at the shared initialization (). We compare held-out target losses after and , where and are normalized LoRA-B gradients. We will also define the source-only and additional-target references.
- Stored-signature alignment (l. 300): Cosine similarity between precomputed task signatures, distinguished from similarity between sampled batch gradients. We will clarify the source-level aggregation used to compute Spearman correlations with observed loss reductions.
- Algorithms 1–2, historical records: Each task contributes a record , containing its signature, replay memory, and adapter snapshot. We will explicitly specify that the first task is trained without replay or KD.
- Replay ratio (): The probability of sampling a historical replay batch at each training step, set to .
- Routing weights and temperature (): , with . The weights determine the historical task sampling distribution.
- Eq. (5), and : denotes the model's next-token predictive distribution, and is a frozen historical adapter. KD uses teacher-to-student KL over the shared vocabulary at temperature , averaged over non-padding prompt and answer tokens, with scaling.
- Era-matched references (l. 371): Replay batches from task use the adapter snapshot saved after that task. Current-task batches use the historical snapshot with the highest routing weight.
Rebuttal by Authors (3/3)
W5. Accuracy-Based Transfer Evaluation
We thank the reviewer for suggesting an accuracy-based evaluation of the transfer matrix. Table 1 uses held-out target loss because it captures changes in predictive confidence even when classification accuracy remains unchanged. We agree that accuracy provides an important complementary measure of downstream transfer. We therefore report the corresponding accuracy gains for all five classification targets below.
Accuracy gain (percentage points) relative to target-only training. Rows denote target tasks and columns denote source tasks.
| Target ↓ / Source → | C-STANCE | FOMC | MeetingBank | Py150 | ScienceQA | NumGLUE-cm | NumGLUE-ds | 20Minuten |
|---|---|---|---|---|---|---|---|---|
| C-STANCE | — | +1.0 | −1.4 | −0.6 | +0.5 | −0.2 | −1.0 | −1.5 |
| FOMC | +1.3 | — | +0.4 | +0.1 | +0.5 | −0.2 | +0.8 | +0.1 |
| ScienceQA | +0.2 | +2.4 | +0.4 | +0.6 | — | +1.0 | +0.8 | −0.2 |
| NumGLUE-cm | +0.7 | +1.2 | +1.6 | +2.3 | +0.5 | — | +4.0 | +0.2 |
| NumGLUE-ds | −0.2 | +0.3 | +0.5 | +1.6 | +0.1 | +2.1 | — | +0.4 |
Agreement between loss and accuracy. To assess whether the two metrics lead to similar source choices, we compare their best sources and compute Spearman correlations across source rankings for each target.
| Target | Loss-best source | Accuracy-best source | Spearman |
|---|---|---|---|
| C-STANCE | FOMC | FOMC | +0.94 |
| FOMC | C-STANCE | C-STANCE | +0.54 |
| ScienceQA | FOMC | FOMC | +0.96 |
| NumGLUE-cm | MeetingBank | NumGLUE-ds | +0.79 |
| NumGLUE-ds | NumGLUE-cm | NumGLUE-cm | +0.68 |
The loss-optimal source also maximizes accuracy for four of five targets, and the source rankings exhibit positive Spearman correlations ranging from 0.68 to 0.96. However, the metrics are not interchangeable: some sources reduce held-out loss without improving accuracy, and the optimal source differs for NumGLUE-ds.
These results indicate that held-out loss captures meaningful source-dependent transfer effects that are largely reflected in downstream accuracy, while also revealing differences that accuracy alone may not capture.
W6. Gradient Signature Reliability
Please see Global Response 4: Reliability of Gradient Signatures, where we quantify selection regret and compare signature-based source selection with oracle and random selection.
W7. DER++ Baseline Correctness
Please see Global Response 2: DER++ Baseline Correctness, where we report the corrected DER++ results following an implementation audit and hyperparameter sweep.
W8. Selection Replaces Curriculum
We agree that this statement was too strong. Curriculum design concerns the ordering of training experiences, whereas source selection determines how historical tasks contribute to subsequent learning. Our experiments do not establish that source selection eliminates task-order effects or replaces curriculum design. We will remove the statement “Selection replaces curriculum” and clarify that TSR complements curriculum learning by controlling the reuse of historical information.
Rebuttal by Authors (1/3)
We thank the reviewer for the thoughtful comments. We address each concern below.
W1. Low-data Overfitting Confound
We agree that the observed transfer gains may partly reflect reduced overfitting, and that regularization and knowledge transfer are not mutually exclusive. We conducted additional experiments to quantify overfitting and examine whether the benefits of compatible replay can be explained by generic regularization alone.
Overfitting under limited supervision. Across five classification targets, training on 50 target examples reaches nearly 100% training accuracy, while test accuracy ranges from 14.4% to 58.9%. Halving the training steps yields negligible improvements, whereas increasing the target budget to 500 examples improves test accuracy by 4.7–11.7 percentage points. These results confirm substantial overfitting in the low-data setting.
Controlling for generic regularization. We compare replay from highly compatible sources with replay from low-compatibility sources and shuffled versions of the selected sources, using matched training budgets. High-compatibility replay improves target test accuracy by an average of 2.06 percentage points over target-only training, compared with 0.06 points for shuffled replay and −0.08 points for low-compatibility replay. It outperforms shuffled replay on all five targets and low-compatibility replay on four of five, with average advantages of 2.00 and 2.14 points, respectively.
Interpretation. These results suggest that the benefit of compatible replay cannot be fully explained by additional training data or generic regularization. Preserving meaningful source supervision and selecting compatible sources both matter for target generalization. Although source-dependent regularization may still contribute, the results support gradient compatibility as a useful signal for identifying beneficial historical sources.
W2. KD Confound
Please see Global Response II, Section 2 (Independent Contribution of Source Selection), where we report a controlled ablation comparing uniform and signature-weighted replay with and without KD. The results show that source selection provides additional gains under matched KD configurations.
W3. Three Conditions and Theoretical Novelty
We agree that our presentation may overstate the theoretical status of the three conditions. We do not claim that headroom, persistence, and compatibility form a universal set of necessary or sufficient conditions for forward transfer. Rather, they are empirically motivated factors that help characterize transfer in low-budget sequential LLM adaptation.
Specifically, headroom describes the opportunity for improvement under limited target supervision, persistence concerns whether benefits from historical information survive continued optimization, and compatibility characterizes which historical sources are beneficial. Our controlled experiments motivate these factors: target-data scaling reveals remaining learning opportunities; carrier comparisons show that initialization benefits diminish while replay sustains gains under our protocol; and pairwise transfer measurements demonstrate substantial variation across sources, including negative transfer.
We also acknowledge important limitations. Target-data headroom is not a strict upper bound on gains from complementary sources, replay is not the only possible mechanism for preserving prior knowledge, and gradient similarity does not guarantee beneficial transfer. We will therefore revise Eq. (2) and Sections 4.1–4.3 to present the three factors as empirical diagnostics and design principles, rather than a necessary and sufficient characterization of forward transfer.
For a detailed discussion of novelty and our distinctions from prior gradient-based methods, including MIR, MER, and Efficient MER, please see Global Response 1: Novelty and Relation to Prior Work.