acceptodds
Under review as a conference paper at ICLR 2027

RASA: Retention-Aware Stream Adaptation for Online Continual Learning in Large Language Models

Abstract

Continual learning (CL) enables large language models (LLMs) to learn new tasks while preserving previously acquired knowledge. Expansion-based methods extend the model through task-specific modules, with low-rank adaptation (LoRA) enabling parameter-efficient expansion. However, in online continual learning (OCL) settings where data arrive in a stream and cannot be revisited, current-task information alone is insufficient to evaluate how adaptation affects performance on previous tasks. In this work, we propose Retention-Aware Stream Adaptation (RASA), a probabilistic framework that combines online statistics of parameter changes with historical feature information. RASA models parameter changes induced by the optimization process as a local Gaussian transition distribution, whose covariance is matched to a reference covariance estimated from the stream to determine adaptive time steps. To incorporate retention, it uses aggregated feature second moments from previous tasks to constrain cumulative changes in LoRA linear outputs on historical input features. A local generalized posterior then combines this constraint with gradient information from the current batch for parameter estimation. The resulting procedure can be incorporated into existing training procedures, reusing available gradients and stored feature statistics without replay or additional forward or backward passes over the data. Experimental results demonstrate that RASA consistently improves the average performance of existing expansion-based methods in OCL.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
online continual learning, large language models, model expansion, covariance matching, knowledge retention

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_9WUmOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies training methods for large language models in the setting of online continual learning, where the data stream is visible in a single pass, old samples are not stored, and task boundaries are visible only during training. The RASA method proposed in the paper can be combined with many existing methods to optimize the training procedure. RASA runs after the optimizer generates the shifts and before submitting the parameters, without modifying the gradients of the original method or the optimizer.

The method can be decomposed into three parts: online covariance matching, which balances the update step of the optimizer; feature-moment retention constraint and correction, which uses second-moment information of historical inputs to control and correct the updates accumulated in the current task; and posterior-based candidate selection, which selects the direction in which the model actually updates. Experiments show that, when combined with seven existing baseline methods, RASA yields improvements in final average performance on all four benchmarks and reduces forgetting in most cases.

Critical strengths

This paper adjusts the update step size during training by solving a covariance matching problem. Then it uses the second-order moments of historically accumulated inputs to evaluate and correct the cumulative effect of new task updates on the model's outputs for historical inputs, thereby learning new tasks while reducing the forgetting of old knowledge. This method does not need to store historical samples, and its overhead is independent of the number of tasks. The paper has a relatively complete theoretical analysis, and experimental results show that the method can be combined with many existing LoRA methods.

Critical weaknesses
  1. The method requires task boundaries during training, which limits its practical value.
  2. Although the proposed method does not need to store historical samples for replay, replay-based methods remain a typical and highly effective class of approaches in continual learning and should not be completely ignored. Moreover, the accuracy of the LLM tested directly without any fine-tuning should also be reported as a naive baseline result.
Questions for authors

Q1 [Clarification]. The method depends on training-time task boundaries. Is this a common, realistic problem setting, and how sensitive is RASA to misspecified boundaries?

Q2 [Suggestion]. Replay-based methods are excluded by protocol, yet they remain a typical and strong family in continual learning. Could the authors add replay-based methods (e.g., ER, Chaudhry et al., 2019; XDER, Boschini et al., 2022) under the same budget as references, and additionally report zero-shot LLM results on the benchmarks?

4 comments

Thank you for the thoughtful review. We clarify the problem setting and address the requested reference evaluation below.

W1. Training-time boundary information and applicability.

Following O-LoRA and GainLoRA, we use task information during training and assume task-agnostic inference. The revised abstract clarifies that task boundaries are unavailable at inference. In practice, users may switch tasks during a conversation without signaling the transition or providing a task ID. We therefore assume that neither task identities nor task boundaries are available at inference.

During training, we assume that the training pipeline observes task boundaries (Sections 3.1 and 4.1). For example, when a data provider delivers successive updates from distinct sources, the pipeline can use the transition between sources as a task boundary. The data delivery process can thus supply boundary information during training without requiring users to provide it at deployment.

W2. Replay-based comparisons and the unadapted LLM reference.

As the reviewer notes, replay-based methods are important and effective approaches in continual learning. However, in some settings, privacy requirements or storage constraints make it difficult to retain or reuse past examples. We study the extent to which a model can maintain performance on previously learned tasks in such settings while learning from each new example only once. To this end, we adopt a single-pass setting in which each example is used only once and both previously used examples and generated replay samples are excluded from training (Sections 3.1 and 4.1).

To clarify how RASA relates to this line of work, we added a discussion of ER, X-DER, and related replay methods to Appendix C (Additional Related Works). We explain how these methods differ from RASA’s no-replay setting in their access to past data. We also cite ER and X-DER where we introduce replay-based methods in the second paragraph of the Introduction (Section 1). In our response to Q2, we report the requested zero-shot results as a baseline for performance without fine-tuning.

Q1. Is the boundary-aware setting realistic, and how sensitive is RASA to misspecified boundaries?

W1 explains the practical distinction between training-time boundary information and task-agnostic inference. We tested boundary-timing sensitivity on Long-sequence CL Order 4 with Meta-Llama-3.1-8B-Instruct (three seeds), shifting supplied boundaries by one optimizer step earlier or later while keeping logical batches unchanged. The table reports AP.

MethodCleanEarly (−1 step)Late (+1 step)
O-LoRA63.82 ± 0.0561.42 ± 0.2961.59 ± 0.46
O-LoRA + RASA70.93 ± 0.1970.43 ± 0.2170.63 ± 0.40
Gain-OLoRA61.04 ± 0.0559.70 ± 0.4959.94 ± 0.25
Gain-OLoRA + RASA71.96 ± 0.1470.92 ± 0.2770.74 ± 0.37
SeqLoRA51.69 ± 0.2450.80 ± 0.5351.16 ± 0.55
SeqLoRA + RASA73.74 ± 0.2272.11 ± 0.3572.60 ± 0.53

RASA retains higher AP than each baseline under both boundary shifts. Its AP decreases by 0.30–1.63 points relative to clean boundaries, while its advantage over the corresponding baseline remains 9.01–21.44 points. Boundary-noise results are added to Appendix B.12 (Boundary Noise Sensitivity), Table B.17.

Q2. Can replay-based references under a matched budget and zero-shot LLM results be added?

For replay-based comparisons, please see W2. Appendix C (Additional Related Works) now includes a discussion addressing this concern and explaining RASA's no-replay setting. We conducted the requested zero-shot evaluation using the same starting checkpoints, evaluation prompts, and metrics as the main experiments, without additional fine-tuning.

BenchmarkLlama-3.1-8B-Instruct, no fine-tuningQwen3-4B, no fine-tuning
Standard CL21.935.23
Long-sequence CL11.4211.89
SuperNI34.6935.29
TRACE38.6139.10
BenchmarkLlama-2-7B-Chat, no fine-tuningQwen3-8B, no fine-tuning
Standard CL17.087.85
Long-sequence CL26.0421.53
SuperNI30.2239.04
TRACE26.7437.50

Added zero-shot references for the two primary backbones to Table 1 and Tables B.6–B.7. Table 1 averages their AP; the captions clarify the zero-shot evaluation.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

Dear Reviewer 9WUm,

Thank you again for taking the time to review our paper. As the discussion period draws to a close, we wanted to follow up on our rebuttal. If you have a chance to read our response, we’d appreciate hearing whether it addresses your concerns. We’d be happy to clarify any remaining points.

Best regards, The Authors

Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts in addressing my concerns. The added zero-shot results, the boundary-noise study, and the expanded discussion of replay-based methods address my main questions.

Overall, I have decided to raise my score from 2 to 3.

Reviewer_vyJEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Retention-Aware Stream Adaptation (RASA) for single-pass, replay-free continual learning of LLMs. Training task boundaries are observed, while task identities are unavailable at inference.

RASA combines three components: adaptive update scaling based on displacement-residual statistics, a task-anchored constraint on cumulative linear-output changes evaluated using historical feature second moments, and selection between corrected optimizer and preconditioned cross-entropy candidates using a local quadratic surrogate. These operations reuse available gradients and retained statistics without additional forward or backward passes over training data.

Experiments evaluate RASA across seven underlying methods and four benchmarks, with additional backbones and full-data experiments in the appendix. The reported results show consistent improvements in final average performance and generally reduced forgetting.

Critical strengths
  1. Practical retention mechanism. RASA constrains cumulative linear-output changes using historical feature statistics, without replay or additional data passes.

  2. Consistent empirical improvements. Gains extend across multiple underlying methods, backbones, and benchmarks, including full-data training.

  3. Informative component analyses. The acquisition/BWT decomposition and historical-versus-isotropic comparison provide useful evidence for the complementary roles of adaptation and retention.

Critical weaknesses
  1. The advantage of covariance matching over simpler adaptive-scaling alternatives is not fully isolated. Figure 3 usefully evaluates sensitivity to the nominal learning rate. However, it does not establish whether the proposed scaling rule outperforms simpler adaptive alternatives when the retention constraint, correction, and candidate selection are held fixed. This distinction matters because scaling alone accounts for substantial improvements in Table 2: for example, SeqLoRA gains 11.17 AP points with A, compared with 12.70 for the complete method. A targeted comparison would clarify how much benefit depends on the particular covariance-matching construction.

  2. The connection between the stored-feature budget and actual retention remains incompletely characterized. The constraint controls linear-output changes on historical features recorded during previous training. It does not capture subsequent feature drift, nonlinear propagation, or routing changes, as the authors appropriately acknowledge. The reported performance and moment ablations support its usefulness, but do not directly establish when the surrogate remains informative as the network evolves. This is particularly relevant to shared-adapter training and methods with trainable routing, where the relationship between the constrained branch response and the full model’s behavior is less direct.

Questions for authors
  1. Adaptive-scaling comparison: Holding the retention constraint, correction, and candidate-selection procedure fixed, how does covariance matching compare with a simpler adaptive scaling rule, such as normalization based on a running update-magnitude statistic? A representative controlled experiment would help distinguish the benefit of adaptive scaling generally from that of the proposed construction.

  2. Retention-surrogate validity: Have the authors measured how the retention budget relates to actual previous-task loss or performance degradation as historical features drift? An analysis on a shared-adapter or routing-based method would help clarify the conditions under which the constraint remains informative.

3 comments

Thank you for the important questions on adaptive scaling and retention. We address each below.

W1. Covariance matching and alternative adaptive-scaling rules.

Table 2 quantifies A’s contribution: SeqLoRA AP increases from 58.02 to 69.19 with A and to 70.72 with full RASA. Figure 3 further compares covariance matching with αₜ=1 while holding R/P fixed across nominal LRs.

Covariance matching uses deviations from the task-local update mean; magnitude normalization uses raw update magnitude. The controlled comparison in Q1 shows that covariance matching achieves higher AP, higher BWT, and lower FR than magnitude normalization on both tested orders, with R/P held fixed. This comparison is now included in Section 4.7, Table 3, with the control definition in Appendix B.8.

W2. The stored-feature retention budget and feature drift.

The guarantee bounds linear-output changes from the task-start anchor on stored historical features (Section 3.3; Proposition A.2, Appendix A.5). This result clarifies what RASA’s retention constraint protects. Specifically, we show that satisfying the budget computed from stored feature statistics bounds the normalized root-mean-square change in the branch’s linear output on historical features relative to the task-start anchor, without replaying past data. It remains valid for those recorded features as representations evolve, but does not guarantee end-to-end loss or accuracy under feature and routing drift.

Table B.14 shows the empirical benefit with A/P fixed (Llama/Gain-OLoRA, Long-sequence Order 4):

Retention conditionAcquisition_old ↑BWT ↑AP_old ↑
No retention constraint74.33 ± 0.56−8.22 ± 0.0966.12 ± 0.64
Isotropic moments, ρ=0.0573.26 ± 0.54−2.68 ± 0.0370.58 ± 0.54
Historical moments, ρ=0.0575.27 ± 0.41−1.54 ± 0.0273.73 ± 0.42

All metrics exclude the last task; values are mean ± sample SD over three seeds. Historical moments improve BWT and AP_old over both controls. Q2 provides complementary measurements of feature drift and old-task loss, illustrating both the empirical benefit and the scope of the stored-feature guarantee.

Q1. How does covariance matching compare with a running update-magnitude normalization rule when R and P are fixed?

We compared three scaling rules using SeqLoRA with Meta-Llama-3.1-8B-Instruct on Standard CL (Order 1) and Long-sequence CL (Order 4). R/P, retention correction, and the remaining training settings were held fixed, using the same clipping policy. Magnitude normalization scales updates using the ratio of running RMS magnitude to current magnitude; covariance matching instead uses deviations from the task-local update mean.

OrderScaling ruleAP ↑BWT ↑FR ↓
1Fixed α=141.43 ± 0.36−45.33 ± 0.1145.57 ± 0.17
1Magnitude normalization41.00 ± 0.35−42.20 ± 0.1843.60 ± 0.18
1Covariance matching (RASA)46.73 ± 0.50−33.87 ± 0.1335.23 ± 0.15
4Fixed α=168.63 ± 0.54−13.13 ± 0.1214.13 ± 0.22
4Magnitude normalization68.40 ± 0.45−11.04 ± 0.0313.65 ± 0.17
4Covariance matching (RASA)73.74 ± 0.22−9.69 ± 0.0911.51 ± 0.11

Values are mean ± sample SD over three seeds. Covariance matching improves AP over magnitude normalization by 5.73 points on Order 1 and 5.34 points on Order 4, with better BWT and FR in both cases. Magnitude normalization improves retention over fixed scaling but does not improve AP. These results support the value of task-centered update fluctuations beyond controlling raw update magnitude in the tested settings. Added to Section 4.7, Table 3, with the magnitude-normalization definition in Appendix B.8.

Q2. How does the retention budget relate to actual previous-task degradation as historical features drift?

We compared RASA with no-R (A/P retained) using Llama-3.1-8B-Instruct and Gain-OLoRA on Long-sequence Order 4. Midpoint and end checkpoints of tasks k=5,10,15 were evaluated offline on 32 fixed held-out examples from each of old tasks 1 and k−1. A ratio 2D/ρ²≤1 satisfies the stored-feature budget. Drift averages three measured layers relative to each condition's old-task acquisition checkpoint. ΔNLL is the change in answer-token NLL from that condition's current-task start; positive values indicate degradation.

Task kCheckpointCondition2D/ρ²Mean feature driftMean ΔNLL
5MidRASA0.9999830.036596+0.005502
5Midno-R21.8842670.060080+0.004566
5EndRASA0.9999850.041962−0.004504
5Endno-R41.6535620.109190+0.063544
10MidRASA0.9999770.057587−0.010345
10Midno-R51.4239250.145410−0.006073
10EndRASA0.9999800.053058−0.008576
10Endno-R48.0898980.139424+0.004975
15MidRASA0.9999820.075993−0.029000
15Midno-R123.6296890.195355+0.074489
15EndRASA0.9999810.083391−0.007123
15Endno-R165.7103660.187018+0.055792

RASA satisfies the budget at all six checkpoints, with lower feature drift in all six comparisons and lower ΔNLL in five of six. At the midpoint of task 5, ΔNLL remains positive (+0.005502) despite budget compliance. These measurements support an empirical retention benefit.

Thank you for the additional experiments and clarifications. They address my main concerns, so I will maintain my rating.

Reviewer_dKRJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies online continual learning (OCL) for large language models that adapt with LoRA. The authors observe that freezing old LoRA branches and avoiding replay is not enough: old tasks can still be damaged by additive interference, feature drift across layers, and routing drift. They propose RASA, a wrapper for existing LoRA OCL methods with three components: adaptive step size, hard retention constraint and candidate selection.

Critical strengths
  1. The paper provides consistent empirical validation. And the gains are not confined to weak baselines: Gain-OLoRA on Standard CL improves from 56.35 to 69.38 AP, and OSFT from 39.40 to 53.44.
  2. RASA adds no parameters, no replay, no extra forward/backward passes, and no task ID at inference. It works with expansion methods, shared adapters (SeqLoRA), and subspace-constrained full fine-tuning (OSFT). The retention constraint is data-free after statistics collection, with O(d²/J) storage (J = 4 blocks). Reported overhead is +196–492 ms/step and about 3 GB GPU for Llama-8B—realistic for an 8B training loop.
  3. The retention constraint is well-motivated and well-ablated.
Critical weaknesses
  1. The component analysis (Table 2) shows that the adaptive step A is the main reason for AP enhancement. The retention constraint R mainly reduces FR, and when used alone it can reduce AP.
  2. Task boundaries are assumed known during training. This is standard in LoRA OCL, but it limits the "online" claim and is not stated in the abstract. The per-task reset also means the method is not directly task-free; an EMA variant would be a useful addition.
  3. All LoRA methods use a fixed 5e-4 LR. Since A is an adaptive step multiplier, it may partially compensate for an untuned LR. The LR sweep (Fig. 3) shows RASA > baseline at every LR, which mitigates the concern, but the comparison "best-tuned baseline vs. best-tuned RASA" is missing. The authors should clarify whether the baselines were tuned, and ideally report the best baseline LR vs. the best RASA LR.

Thanks for author's rebuttal, I have no followup quetions on this paper. Though the innovation seems to be mild, the paper is overall complete and easy to read.

Questions for authors

Q1. What fraction of steps have q^(δt)>0\hat{q}(\delta_t) > 0, and what is the distribution of the correction factor c? These would quantify how often R and P actually intervene. Q2. How sensitive is RASA to boundary noise? Can the authors add a boundary-shift or change-point-detection experiment? Q3. Were the baselines' learning rates tuned? What is the best baseline LR vs. the best RASA LR? Q4. Table B.15 does not include OSFT. Since OSFT is the main evidence for "beyond expansion," its overhead should be reported.

4 comments

Thank you for the careful review. We address the component roles, training assumptions, and requested measurements below.

W1. Contributions of adaptive scaling and retention.

R is a retention mechanism designed to complement adaptation; R-only performance does not fully capture its contribution to RASA (section 4.4 Component Analysis, Table 2). Its constraint can reduce forgetting while limiting new-task acquisition, as illustrated in Figure 1. The R-only ablation therefore characterizes this trade-off, while R’s contribution within RASA is assessed by holding the other components fixed.

Adding R to A improves AP and reduces FR for all three methods in Table 2.

MethodA: AP ↑ / FR ↓A+R: AP ↑ / FR ↓Full RASA: AP ↑ / FR ↓
O-LoRA67.48 / 14.4369.09 / 13.3069.61 / 12.58
Gain-OLoRA66.60 / 1.2869.06 / 1.1070.21 / 0.85
SeqLoRA69.19 / 12.6569.96 / 11.8570.72 / 11.90

Table B.14 provides a further control with A and P fixed. For Llama/Gain-OLoRA on Long-sequence Order 4, adding historical retention (ρ=0.05) improves BWT from −8.22 to −1.54 and AP_old from 66.12 to 73.73, while maintaining acquisition (74.33→75.27). These results demonstrate R’s complementary contribution beyond adaptive scaling.

Candidate and correction diagnostics are added to Appendix B.8, Table B.12(c,d).

W2. Training-boundary assumptions and the proposed EMA variant.

During training, RASA observes task boundaries and uses each example once without replay. At inference, neither task identities nor task boundaries are available (Sections 3.1 and 4.1). The revised Abstract now states explicitly that task boundaries are unavailable at inference. We also make the training-boundary assumption explicit in Appendix D to strengthen the soundness of our claims by clarifying the conditions under which they hold.

The task-local displacement mean resets at each boundary, while the reference variance and time-step statistics persist across tasks (Appendices A.4 and B.3). As the reviewer suggests, an EMA variant could track the displacement mean continuously, removing the need for a boundary-triggered reset of that statistic. This offers a new direction for extending RASA to task-free training on real-world streams where neither task identities nor explicit boundaries are available. We thank the reviewer for this insight and have explicitly included the proposed EMA variant as a direction for future work in Appendix D.

W3. Nominal learning-rate selection and the effect of adaptive scaling.

Table B.2 uses a nominal LR of 5×10⁻⁴ for LoRA methods and 10⁻⁵ for OSFT, with the same LR within each baseline–RASA pair. The shared LoRA LR was chosen for strong average AP across the six LoRA baselines. Q3 shows that RASA retains its gains when each variant uses its own best LR in the evaluated grid.

Figure 3 isolates adaptive scaling on Llama/Gain-InfLoRA, Long-sequence Order 4: full RASA achieves higher mean AP than both the baseline and α=1 RASA at every evaluated LR (0.1–10× the reference LR), with R and P fixed between the two RASA variants.

Q1. How often are the raw optimizer update and retention correction consequential?

Added to Appendix B.8: Table B.12(c,d) reports the raw-score frequencies, correction rates, and correction-factor distributions. Measurements use Meta-Llama-3.1-8B-Instruct on Standard CL Orders 1–3 with three seeds. Both candidate directions are used. Table B.12 shows optimizer selection in 68–76% of decisions and CE selection in 23–28%.

MethodOptimizer selected (%)CE selected (%)Rejected (%)
O-LoRA75.52 ± 0.9023.09 ± 0.301.39 ± 0.60
Gain-OLoRA68.40 ± 0.3027.78 ± 0.603.82 ± 0.80
SeqLoRA72.22 ± 1.0825.35 ± 0.802.43 ± 1.59

Values are mean ± sample SD across seeds, after averaging orders within each seed, using 576 decisions per method (192 per seed). Raw optimizer scores are positive in 13–15% of observations, and retention correction is regularly active for both directions. Scores are measured after native projection (optimizer) or norm capping (CE), before RASA scaling/correction.

Method + RASARaw optimizer q̂>0 (%)Raw CE q̂>0 (%)Optimizer candidate c<1 (%)CE candidate c<1 (%)
O-LoRA13.890.0040.74 ± 2.8946.06 ± 2.44
Gain-OLoRA14.930.0055.09 ± 3.5065.05 ± 2.12
SeqLoRA12.670.0020.60 ± 3.8230.09 ± 2.81

Raw-score percentages pool 576 observations per method. Correction rates cover 432 historical candidates per direction, including unselected candidates, and report mean ± sample SD across seeds.

When correction is applied, median c is approximately 0.77–0.94 for the expansion methods and 0.50 for SeqLoRA. The table reports pooled median [P5, P95]. All-historical distributions include c=1; corrected subsets contain only c<1.

Method / correction mapCandidateAll historical candidates: median [P5, P95]Corrected N / 432Corrected subset: median [P5, P95]
O-LoRA / expansionOptimizer1.0000 [0.9160, 1.0000]176 / 4320.9407 [0.9064, 0.9754]
O-LoRA / expansionCE1.0000 [0.8129, 1.0000]199 / 4320.9150 [0.7984, 0.9914]
Gain-OLoRA / expansionOptimizer0.9069 [0.7845, 1.0000]238 / 4320.9053 [0.7830, 0.9544]
Gain-OLoRA / expansionCE0.8748 [0.6396, 1.0000]281 / 4320.7742 [0.6349, 0.9292]
SeqLoRA / shared adapterOptimizer1.0000 [0.5000, 1.0000]89 / 4320.5000 [0.2500, 0.5000]
SeqLoRA / shared adapterCE1.0000 [0.5000, 1.0000]130 / 4320.5000 [0.2500, 0.5000]

Q2. How sensitive is RASA to boundary noise?

We evaluated AP on Long-sequence CL Order 4 with Meta-Llama-3.1-8B-Instruct (three seeds), shifting supplied boundaries by one optimizer step earlier or later while keeping logical batches unchanged.

MethodCleanEarly (−1 step)Late (+1 step)
O-LoRA63.82 ± 0.0561.42 ± 0.2961.59 ± 0.46
O-LoRA + RASA70.93 ± 0.1970.43 ± 0.2170.63 ± 0.40
Gain-OLoRA61.04 ± 0.0559.70 ± 0.4959.94 ± 0.25
Gain-OLoRA + RASA71.96 ± 0.1470.92 ± 0.2770.74 ± 0.37
SeqLoRA51.69 ± 0.2450.80 ± 0.5351.16 ± 0.55
SeqLoRA + RASA73.74 ± 0.2272.11 ± 0.3572.60 ± 0.53

RASA retains higher AP than each baseline under both boundary shifts. Its AP decreases by 0.30–1.63 points relative to clean boundaries, while its advantage over the corresponding baseline remains 9.01–21.44 points. These results concern ±1-step timing errors within the observed-boundary setting. Added to Appendix B.12 (Boundary Noise Sensitivity), Table B.17.

Q3. Were learning rates tuned, and what are the best baseline and RASA learning rates?

Yes, for the LoRA methods. We initially evaluated {1×10⁻⁵, 5×10⁻⁵, 1×10⁻⁴, 5×10⁻⁴, 1×10⁻³} on Long-sequence CL Order 4 with Meta-Llama-3.1-8B-Instruct. We chose 5×10⁻⁴ for its strong average AP across the six LoRA baselines and fixed it for subsequent LoRA experiments. For OSFT and OSFT+RASA, we used 10⁻⁵, following the decoder-only setting in the original OSFT paper, as documented in our Table B.2. Each row uses the LR with the highest mean AP across three seeds for that variant; FR is reported at the same LR.

MethodBest evaluated LRAP ↑FR ↓
Gain-InfLoRA1×10⁻³56.46 ± 0.293.08 ± 0.14
Gain-InfLoRA + RASA5×10⁻⁴70.35 ± 0.631.02 ± 0.01
O-LoRA1×10⁻⁴65.38 ± 0.5719.34 ± 0.06
O-LoRA + RASA5×10⁻⁴70.93 ± 0.1913.16 ± 0.12
InfLoRA5×10⁻⁵77.75 ± 0.288.49 ± 0.14
InfLoRA + RASA5×10⁻⁵80.52 ± 0.247.21 ± 0.28

RASA improves AP by 13.89, 5.55, and 2.77 points for Gain-InfLoRA, O-LoRA, and InfLoRA, respectively, with lower FR in all three comparisons. For the remaining three LoRA methods—SeqLoRA, IncLoRA, and Gain-OLoRA—both baseline and RASA achieve their highest AP at 5×10⁻⁴. The shared LoRA LR also reflects sensitivity across methods: SeqLoRA reaches only 24.16 AP at 1×10⁻⁴ and fails to train successfully at 1×10⁻³ (AP=0).

Q4. What is the overhead when RASA is applied to OSFT?

RASA increases OSFT training time by 20.0–21.9% and mean peak GPU memory by approximately 30%, while mean peak CPU memory increases by less than 1%. We used the same profiling protocol as Table B.15 on Standard CL (Order 1). Each entry reports OSFT / OSFT + RASA.

BackboneTime (ms/step)Mean GPU peak (GiB)Mean CPU peak (GiB)
Meta-Llama-3.1-8B-Instruct2453.49 / 2944.8631.436 / 40.86020.419 / 20.543
Qwen3-4B1835.89 / 2237.4230.676 / 39.84818.722 / 18.824

Added both OSFT rows to Table B.15 and updated the overhead analysis in Appendix B.10.

AuthorsOpenReview ↗

A Brief Follow-up Before the Discussion Ends

Dear Reviewer dKRJ,

Thank you again for your thoughtful review. With the discussion period coming to an end, we wanted to check in about our response above. If there are any points you’d like us to explain further, please let us know. We’d be glad to provide additional clarification.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the discussion and revisions

We thank all three reviewers for their careful reading of our paper and their thoughtful feedback. Our individual responses address each of the points they raised. We are grateful to Reviewer dKRJ for revisiting the paper and noting that they had no further questions, and to Reviewer vyJE for confirming that our additional experiments and clarifications addressed their main concerns.

  • Reviewer dKRJ asked how much of the gain comes from adaptive scaling. Adding the retention constraint to scaling improved average performance (AP) and reduced forgetting for all three methods in Table 2. RASA also retained its gains when each method used its own best learning rate from the grid. We added the requested correction statistics and OSFT overhead measurements to Tables B.12 and B.15.
  • Reviewer vyJE asked about simpler scaling rules and the effect of feature drift. With the other components held fixed, covariance matching improved AP over running magnitude normalization by more than five points on both task orders tested (Table 3). In a separate analysis, enabling the retention constraint reduced feature drift at all six checkpoints and yielded a more favorable change in held-out old-task loss at five of six.

We also thank Reviewer 9WUm for raising three questions about the evaluation:

  • Replay baselines. Replay-based methods such as ER and X-DER are important baselines in continual learning, and we appreciate the reviewer’s suggestion to include them. Our study, however, focuses on a single-pass, replay-free setting (§3.1) in which each training example is used only once and neither stored nor generated examples can be replayed. Even with the same memory budget, ER and X-DER can revisit earlier examples during later updates, whereas RASA uses aggregate feature statistics to guide those updates. We therefore evaluated all methods under the same replay-free protocol and expanded Appendix C to clarify how ER and X-DER relate to this setting.
  • Training-time task boundaries. We clarified how a training pipeline can observe a boundary, for example when a data provider switches sources. We also shifted each boundary one optimizer step early or late. Across the three methods tested, RASA retained a 9.0–21.4 point AP advantage over the corresponding baselines under these shifts (Table B.17).
  • Zero-shot performance. We evaluated all four backbones before fine-tuning on all four benchmarks, using the same prompts and metrics as in the main experiments. Table 1 now includes the zero-shot results for the two primary backbones alongside the continual-learning results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.