- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies training methods for large language models in the setting of online continual learning, where the data stream is visible in a single pass, old samples are not stored, and task boundaries are visible only during training. The RASA method proposed in the paper can be combined with many existing methods to optimize the training procedure. RASA runs after the optimizer generates the shifts and before submitting the parameters, without modifying the gradients of the original method or the optimizer.
The method can be decomposed into three parts: online covariance matching, which balances the update step of the optimizer; feature-moment retention constraint and correction, which uses second-moment information of historical inputs to control and correct the updates accumulated in the current task; and posterior-based candidate selection, which selects the direction in which the model actually updates. Experiments show that, when combined with seven existing baseline methods, RASA yields improvements in final average performance on all four benchmarks and reduces forgetting in most cases.
Critical strengths
This paper adjusts the update step size during training by solving a covariance matching problem. Then it uses the second-order moments of historically accumulated inputs to evaluate and correct the cumulative effect of new task updates on the model's outputs for historical inputs, thereby learning new tasks while reducing the forgetting of old knowledge. This method does not need to store historical samples, and its overhead is independent of the number of tasks. The paper has a relatively complete theoretical analysis, and experimental results show that the method can be combined with many existing LoRA methods.
Critical weaknesses
- The method requires task boundaries during training, which limits its practical value.
- Although the proposed method does not need to store historical samples for replay, replay-based methods remain a typical and highly effective class of approaches in continual learning and should not be completely ignored. Moreover, the accuracy of the LLM tested directly without any fine-tuning should also be reported as a naive baseline result.
Questions for authors
Q1 [Clarification]. The method depends on training-time task boundaries. Is this a common, realistic problem setting, and how sensitive is RASA to misspecified boundaries?
Q2 [Suggestion]. Replay-based methods are excluded by protocol, yet they remain a typical and strong family in continual learning. Could the authors add replay-based methods (e.g., ER, Chaudhry et al., 2019; XDER, Boschini et al., 2022) under the same budget as references, and additionally report zero-shot LLM results on the benchmarks?
4 comments
Q1. Is the boundary-aware setting realistic, and how sensitive is RASA to misspecified boundaries?
W1 explains the practical distinction between training-time boundary information and task-agnostic inference. We tested boundary-timing sensitivity on Long-sequence CL Order 4 with Meta-Llama-3.1-8B-Instruct (three seeds), shifting supplied boundaries by one optimizer step earlier or later while keeping logical batches unchanged. The table reports AP.
| Method | Clean | Early (−1 step) | Late (+1 step) |
|---|---|---|---|
| O-LoRA | 63.82 ± 0.05 | 61.42 ± 0.29 | 61.59 ± 0.46 |
| O-LoRA + RASA | 70.93 ± 0.19 | 70.43 ± 0.21 | 70.63 ± 0.40 |
| Gain-OLoRA | 61.04 ± 0.05 | 59.70 ± 0.49 | 59.94 ± 0.25 |
| Gain-OLoRA + RASA | 71.96 ± 0.14 | 70.92 ± 0.27 | 70.74 ± 0.37 |
| SeqLoRA | 51.69 ± 0.24 | 50.80 ± 0.53 | 51.16 ± 0.55 |
| SeqLoRA + RASA | 73.74 ± 0.22 | 72.11 ± 0.35 | 72.60 ± 0.53 |
RASA retains higher AP than each baseline under both boundary shifts. Its AP decreases by 0.30–1.63 points relative to clean boundaries, while its advantage over the corresponding baseline remains 9.01–21.44 points. Boundary-noise results are added to Appendix B.12 (Boundary Noise Sensitivity), Table B.17.
Q2. Can replay-based references under a matched budget and zero-shot LLM results be added?
For replay-based comparisons, please see W2. Appendix C (Additional Related Works) now includes a discussion addressing this concern and explaining RASA's no-replay setting. We conducted the requested zero-shot evaluation using the same starting checkpoints, evaluation prompts, and metrics as the main experiments, without additional fine-tuning.
| Benchmark | Llama-3.1-8B-Instruct, no fine-tuning | Qwen3-4B, no fine-tuning |
|---|---|---|
| Standard CL | 21.93 | 5.23 |
| Long-sequence CL | 11.42 | 11.89 |
| SuperNI | 34.69 | 35.29 |
| TRACE | 38.61 | 39.10 |
| Benchmark | Llama-2-7B-Chat, no fine-tuning | Qwen3-8B, no fine-tuning |
|---|---|---|
| Standard CL | 17.08 | 7.85 |
| Long-sequence CL | 26.04 | 21.53 |
| SuperNI | 30.22 | 39.04 |
| TRACE | 26.74 | 37.50 |
Added zero-shot references for the two primary backbones to Table 1 and Tables B.6–B.7. Table 1 averages their AP; the captions clarify the zero-shot evaluation.
Gentle Follow-up on Our Rebuttal
Dear Reviewer 9WUm,
Thank you again for taking the time to review our paper. As the discussion period draws to a close, we wanted to follow up on our rebuttal. If you have a chance to read our response, we’d appreciate hearing whether it addresses your concerns. We’d be happy to clarify any remaining points.
Best regards, The Authors
Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts in addressing my concerns. The added zero-shot results, the boundary-noise study, and the expanded discussion of replay-based methods address my main questions.
Overall, I have decided to raise my score from 2 to 3.
Thank you for the thoughtful review. We clarify the problem setting and address the requested reference evaluation below.
W1. Training-time boundary information and applicability.
Following O-LoRA and GainLoRA, we use task information during training and assume task-agnostic inference. The revised abstract clarifies that task boundaries are unavailable at inference. In practice, users may switch tasks during a conversation without signaling the transition or providing a task ID. We therefore assume that neither task identities nor task boundaries are available at inference.
During training, we assume that the training pipeline observes task boundaries (Sections 3.1 and 4.1). For example, when a data provider delivers successive updates from distinct sources, the pipeline can use the transition between sources as a task boundary. The data delivery process can thus supply boundary information during training without requiring users to provide it at deployment.
W2. Replay-based comparisons and the unadapted LLM reference.
As the reviewer notes, replay-based methods are important and effective approaches in continual learning. However, in some settings, privacy requirements or storage constraints make it difficult to retain or reuse past examples. We study the extent to which a model can maintain performance on previously learned tasks in such settings while learning from each new example only once. To this end, we adopt a single-pass setting in which each example is used only once and both previously used examples and generated replay samples are excluded from training (Sections 3.1 and 4.1).
To clarify how RASA relates to this line of work, we added a discussion of ER, X-DER, and related replay methods to Appendix C (Additional Related Works). We explain how these methods differ from RASA’s no-replay setting in their access to past data. We also cite ER and X-DER where we introduce replay-based methods in the second paragraph of the Introduction (Section 1). In our response to Q2, we report the requested zero-shot results as a baseline for performance without fine-tuning.