acceptodds
Under review as a conference paper at ICLR 2027

Scaling Continual Learning to 125 Tasks with Experience Replay

Abstract

The ability to continually acquire new skills while retaining existing capabilities is important for the long-term deployment of vision-language-action (VLA) models. However, scaling VLA continual learning beyond a hundred tasks remains underexplored. Prior work mainly studies task-wise updates on small task sets. Growing task pools and the need to incorporate new capabilities in batches motivate phase-wise continual learning, where each update jointly learns a batch of new tasks. We systematically study which continual learning recipes scale to this setting. Experiments on LIBERO and RoboCasa365 show that task-wise findings do not directly transfer to phase-wise updates. For example, adding low-rank adaptation (LoRA) to experience replay can benefit some task-wise settings through better old-task retention, but this advantage does not reliably persist across phases. Higher ranks and tuned adapter learning rates improve acquisition, yet LoRA still trails full-model ER and can forget more of the capabilities learned in intermediate phases. Further analysis shows that task composition and task scale both shape learning dynamics. We find that simple full-model experience replay (ER) effectively balances new-task acquisition and old-task retention in phase-wise VLA continual learning, making it the most practical choice among the recipes we evaluate. Across four phases and 125 tasks in RoboCasa365, full-model ER continually acquires new capabilities while preserving old ones, using only of each old task's data for replay and requiring neither additional teacher training nor online data collection. Our results establish full-model ER as a practical, scalable recipe for phase-wise VLA continual learning.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Continual learning, Vision Language Action models, Experience Replay

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_diAWOpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies continual learning of a pretrained VLA (\pi 0.5) when each update adds a batch of tasks (phase-wise) rather than one, on LIBERO and RoboCasa365.

Starting from full-model experience replay (ER) with 2% of old-task data, it compares LoRA, model merging, RETAIN, RL and on-policy distillation, with 3 findings:

(1) Task-wise conclusions do not transfer: adding LoRA to ER, a common default, mostly limits acquisition, and tuned high-rank LoRA preserves the initial tasks but forgets later-acquired skills (Phase 2: 34.7% → 27.7%, versus 35.4% → 35.3% for ER). (2) At a fixed budget, larger task groups improve both acquisition and retention, and retention depends strongly on which tasks are grouped. (3) Plain full-model ER, optionally with a short low-learning-rate continuation, approaches separately trained phase teachers without extra teachers or online rollouts.

Critical strengths
  1. Per-phase reporting at meaningful scale reveals a non-obvious effect: full-model ER keeps skills learned in later phases, while tuned LoRA matches it on the initial tasks but loses them (Phase 2 after the final update: 35.3% vs 27.7%; Table 2). So an aggregate forgetting score, dominated by the 65 initial tasks, would hide this.

  2. The comparison is broad and fair. LoRA is swept over rank and learning rate, and ER is compared against merging, RETAIN, RL and distillation, alongside controlled task-composition and update-granularity experiments. This makes full-model ER a credible reference baseline for future phase-wise VLA work.

Critical weaknesses

All large-scale results come from one training seed, one backbone and one task order, and the reported intervals capture only evaluation noise. Large effects, such as LoRA’s intermediate-phase forgetting, look robust, but smaller ones do not: the recommended low-learning-rate continuation (ER+3k) beats plain ER by only 1–2 points, about one interval half-width.

In addition, the paper offers no definitive explanation of why full fine-tuning with ER leads to better results. Does full fine-tuning learn something more generalizable, or does it simply produce sparser updates, which LoRA disrupts? I would like to see some reasons behind this behavior.

As for the claim that high-rank LoRA approaches full fine-tuning’s training compute, FLOPs are not the most relevant cost. LoRA mainly saves gradient and optimizer-state memory and reduces gradient communication across GPUs, which can lower hardware requirements and wall-clock time for a model of this size. Measured memory and training time would make the comparison more convincing.

Questions for authors

Do you have some deeper insights regarding the full parameter ER generalizing better? Or is it task/order specific?

Are measured peak device memory and wall-clock times available for the main full-model ER and LoRA 5× runs? If not, will you limit the efficiency comparison to the reported FLOP estimates?

2 comments

We thank the reviewer for recognizing that our phase-wise reporting reveals intermediate-phase forgetting that aggregate metrics can hide, and for acknowledging the breadth and fairness of our comparisons. We appreciate your questions about the repeatability of continuation gains, measured efficiency, and the reasons for full-model ER's stronger performance.

Note: Changes made specifically in response to Reviewer diAW are highlighted in soft purple (#9370DB; RGB: 147, 112, 219).


Q1: The experiments use one training seed, and the small gains from ER continuation may not be robust.

A: Thank you for raising this concern. Computational constraints limit the main four-phase experiments to a single training seed. To examine the low-learning-rate continuation gain raised in your comment, we repeat the final 3k continuation of ER+3k with seeds 43 and 44. All three runs start from the same checkpoint in the ER+3k stream, after the 15k Phase4 update and before its final 3k continuation, sharing the earlier Phase2 and Phase3 continuations. These results are now included in Appendix B.12, Table 18.

Both additional seeds improve Phase4 success and average success across all 125 tasks. Phase4 success rises from 7.9% to 9.0% and 9.3%, gains of 1.1 and 1.4 percentage points. Both runs also improve Phase2 and Phase3 success. The ER+3k result from Table 2 is included below.

PolicyPhase4 success (%)All 125 tasks (%)
ER+3k starting checkpoint7.944.2
ER+3k8.645.4
Additional continuation, seed 439.045.3
Additional continuation, seed 449.344.8

The additional results are generally within the 95% evaluation intervals of ER+3k in Table 2. Although single-seed results can vary, all three continuation runs consistently improve Phase4 success and average success across all 125 tasks.


Q2: What are LoRA's measured training time and peak device memory costs?

A: Thank you for this suggestion. We have added measured step times and peak memory in Appendix B.7, Table 12. In a controlled benchmark with global batch size 512 and the main experiments' LoRA ranks of 128/256, high-rank LoRA takes longer per step than full-model ER and does not reduce peak device memory.

MethodStep time (s)JAX allocator peak (GiB/device)Device peak (GB/device)Adam state (GB/device)
Full-model ER7.127.549.43.1
LoRA 5×10.926.451.50.8

LoRA does save optimizer-state memory, as the reviewer notes. However, the allocator peak falls only modestly, and the measured device peak is higher. A second benchmark at 16 samples per device gives the same ordering: micro-step wall-clock time is 2.4 versus 3.3 seconds, and device peak memory is 16.7 versus 20.7 GB, for full-model ER and LoRA respectively.

The compiled LoRA step also contains more all-gathers (63→96), reduce-scatters (16→20), and all-to-alls (74→98), while the arithmetic savings at these ranks are modest. Lower-rank LoRA may offer larger efficiency gains, but its acquisition is substantially weaker in our rank ablation. The high-rank configuration that improves acquisition therefore does not retain a clear efficiency advantage in our benchmark.

Q3: Why does full-model ER perform better than LoRA?

A: Thank you for this constructive question. We have added spectral and interference analyses to the main text and Appendix B.9 (Figures 8 and 17). Taking Phase2 as a case study, we decompose checkpoint updates by SVD, examine where the next update places its energy, and test sensitivity to perturbations. The results identify two connected reasons for LoRA's greater intermediate-phase forgetting.

First, LoRA concentrates the changes supporting acquired skills in relatively few directions. Our main LoRA configuration uses rank 128 in the language model and 256 in the action expert, yet the action expert's key/value updates need a median of only 17 singular directions to capture 95% of the update energy, compared with 99 for full-model ER (Figure 8a).

Second, subsequent LoRA learning places more energy in the directions supporting earlier skills. The cumulative Phase2-to-Phase4 LoRA update is also more concentrated and has greater total energy. On the same leading directions of the preceding LoRA update, the next actual LoRA update deposits 6.1× as much energy as the full-model ER update. The KeepLoRA analysis provides complementary evidence: at history threshold 0.99, the Phase2 historical directions alone remove 85.1% of the initial Phase3 gradient energy. Much of the incoming learning signal therefore overlaps the space associated with previous learning.

We then test whether interference in these directions has a larger functional effect. We select the leading directions of the preceding LoRA update within KeepLoRA's principal subspace and compare them with matched directions in the residual subspace. Both receiving policies use the same LoRA-defined directions. In the fixed-input test, perturbations are constructed from the actual next-phase LoRA update, with Frobenius norms matched across subspaces to keep perturbation strength equal. Adding the principal-direction perturbation produces a larger prediction-loss increase for LoRA than for full-model ER. In the rollout test, equal-norm principal-direction perturbations reduce Phase2 success by 6.3 percentage points for LoRA and 0.3 for full-model ER; residual-direction perturbations have little effect on either policy. Perturbations are added to the Phase2 weights in both tests.

Together, the concentration of acquired changes, the stronger later updates along those directions, and their greater sensitivity to interference explain why LoRA loses more intermediate-phase skills in this stream. This closely matches the mechanism reported in Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation (EBLoRA), which studies concentrated updates and their vulnerability in llms.

We also test EBLoRA's energy-balancing factorization during Phase2 learning (Appendix B.9, Figure 18). It produces the intended flat adapter spectrum, but new-task success falls substantially despite training loss similar to standard LoRA. Balancing the spectrum while preserving acquisition therefore requires further study. Together with the measured training costs, these results favor full-model ER over the tested LoRA configurations in our phase-wise setting.

The broader explanation is the change in learning dynamics. In our task-wise comparisons, new-task acquisition is relatively easy and forgetting is the dominant difficulty. With phase-wise updates, learning many new tasks together makes acquisition harder, while full-model ER already preserves old skills well. LoRA's restricted acquisition and concentrated, mutually interfering updates then become more consequential. This protocol dependence appears on both LIBERO and RoboCasa365. The detailed spectral and perturbation analyses explain the RoboCasa365 stream under the task order studied here; they do not establish task-order independence.

We hope these additions address your concerns about continuation robustness, measured efficiency, and the mechanisms behind full-model ER's advantage. We would be happy to discuss further questions and clarify the manuscript accordingly.

Reviewer_BysSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates phase-wise continual learning for VLA models, where multiple new tasks are learned jointly at each update. The authors examine how task composition and update granularity affect continual learning and compare experience replay, LoRA, model merging, reinforcement learning, and policy distillation. Experiments on LIBERO and RoboCasa365 show that findings from task-wise continual learning do not necessarily transfer to phase-wise settings. Across 125 tasks, full-model ER achieves a favorable balance between new-task acquisition and old-task retention using only 2% of each previous task's data, while LoRA exhibits greater forgetting of intermediate-phase capabilities.

Critical strengths
  1. The paper provides a systematic empirical evaluation across multiple continual learning settings and competing methods, offering strong evidence for the effectiveness and practicality of full-model ER in scaling VLA continual learning.

  2. The paper offers valuable insights into how task composition and update granularity influence continual learning. In particular, the analysis of support tasks and task grouping sheds light on the different learning and forgetting dynamics between task-wise and phase-wise settings.

Critical weaknesses
  1. The evaluation does not sufficiently investigate the sensitivity of ER to its replay configuration. Section 3.2 fixes the replay memory at 2% of each old task's data and allocates 31.25% of each training batch to old-task samples. These settings remain unchanged throughout the four-phase experiments, even as the number of previously learned tasks increases. Since replay is central to the paper's main conclusions, ablations varying the memory size and replay sampling proportion would help determine whether the reported acquisition and retention performance is robust to these choices.

  2. The comparison with replay-free continual learning methods remains limited. While the paper evaluates standard LoRA and several replay-based approaches, it does not include dedicated replay-free methods designed to mitigate catastrophic forgetting. Prior VLA continual learning work, such as OrthoSkillVLA [1], has already evaluated replay-free approaches including O-LoRA [2] and KeepLoRA [3]. Given the paper's emphasis on the effectiveness and practicality of ER, comparisons with representative replay-free methods, or a discussion of their applicability to phase-wise continual learning, would help clarify whether strong acquisition and retention can also be achieved without storing previous-task data.

[1] OrthoSkillVLA: Continual Skill Learning via Gradient-Informed Skill Subspace Adaptation. PRCV 2026.

[2] Orthogonal Subspace Learning for Language Model Continual Learning. Findings of EMNLP 2023.

[3] KeepLoRA: Continual Learning with Residual Gradient Adaptation. ICLR 2026.

Questions for authors

Please refer to the critical weaknesses above.

3 comments

We thank the reviewer for recognizing the strong empirical evidence for the effectiveness and practicality of full-model ER, and the insights into task composition, update granularity, and support tasks. We appreciate your constructive suggestions for examining replay configurations and dedicated replay-free alternatives.

Note: Changes made specifically in response to Reviewer BysS are highlighted in teal (#16747D; RGB: 22, 116, 125).


Q1: The sensitivity of ER to its replay configuration is insufficiently explored.

A: Thank you for this constructive suggestion. We have added the requested experiments to the main text (Figure 9). We vary the replay ratio—the fraction of each training batch sampled from old-task data—at fixed 2% memory, and the memory capacity—the fraction of each old task's data stored—at fixed 31.25% replay. Both sweeps start from the same Phase3 checkpoint and use batch size 128 for 8k Phase4 steps to accelerate the experiments.

Replay ratio matters substantially more than memory capacity. A low replay ratio markedly reduces old-task retention, whereas a high ratio sacrifices new-task acquisition. Ratios around 30–50% provide a good balance. Once replay is not too low, retention varies relatively modestly, while allocating more of the batch to old data increasingly restricts acquisition.

Memory capacity has a smaller effect on both retention and acquisition. The smallest tested pool, 0.5%, actually gives slightly higher old- and new-task success than the larger pools. We verified that samples are drawn randomly from each replay pool. These tests cover one Phase3-to-Phase4 update; memory diversity may matter more over longer, multi-phase training. Nevertheless, they show that even a 0.5% memory can preserve substantial old-task performance.

In our main experiment, full-model ER keeps the replay ratio at 31.25% and stores 2% of each old task's data. After Phase4, average success on Phases1–3 is just 0.6 percentage points below the levels reached when those phases were learned (Table 2). Increasing replay as old tasks accumulate may offer only limited additional gains in retention while leaving less room for learning new tasks. With retention already strong, we therefore keep the ratio fixed and consider acquisition the more important challenge in this phase-wise setting.

Q2: The comparison lacks dedicated replay-free methods designed to mitigate forgetting.

A: Thank you for this suggestion. We have added O-LoRA and KeepLoRA experiments in Appendix B.10. All replay-free comparisons start from the same Phase2 checkpoint and train on Phase3 demonstrations without replay. We follow the original designs and adapt their constraints and hyperparameters to the VLA setting.

  • O-LoRA: Following the original method, we keep historical adapter directions fixed and penalize overlap with new query/value adapter directions. In our adaptation, the unnormalized orthogonality penalty overwhelms the action-loss gradient and causes persistent gradient clipping. We normalize it to the fraction of the new adapter's energy in the historical subspace. This keeps the penalty gradient below the action-loss gradient while still strongly reducing update energy in the historical directions (Figure 19). Thus, normalization preserves the intended subspace constraint while making its scale compatible with action learning.

  • KeepLoRA: We follow its residual-subspace update rule, estimating historical directions from Phase2 calibration gradients for the VLA adaptation. We sweep the history-energy threshold over 0.6, 0.8, and 0.99 to examine the retention–acquisition balance. At threshold 0.8, projection removes 88.1% of the initial gradient energy (Figure 20). We therefore also test a 10× adapter learning rate to strengthen learning within the remaining subspace, without relaxing the projection.

Selected success rates (%) are shown below; the complete tuning results are in Tables 15–16 and Figure 21.

MethodPhase3 stepsPhase1: oldPhase2: oldPhase3: new
Standard LoRA, no replay3k33.61.30.0
Full-model SFT, no replay3k41.45.34.7
O-LoRA, normalized, no replay3k39.92.20.0
KeepLoRA, threshold 0.8, LR 1×, no replay15k63.318.84.4
KeepLoRA, threshold 0.8, LR 10×, no replay15k41.47.815.3
ER + LoRA, LR 5×, main experiment15k66.831.114.7

At 3k steps, normalized O-LoRA retains more than standard LoRA, but remains below full-model SFT on both old phases and records no new-task successes. The diagnostics confirm that its intended overlap suppression is active; this does not translate into strong phase-wise learning in our tests, so we did not extend it to the full stream.

KeepLoRA with the larger learning rate achieves the strongest acquisition among our tested replay-free configurations. After 15k steps, it matches the Phase1 retention and exceeds the Phase2 retention of full-model SFT trained for only 3k steps. Its Phase3 acquisition also slightly exceeds the main ER + LoRA 5× result. Increasing the learning rate improves acquisition within the residual subspace, although it clearly weakens retention compared with KeepLoRA at the reference learning rate.

In the LoRA mechanism analysis, we also test EBLoRA's spectrum-balancing technique in a separate comparison with replay in both arms (Appendix B.9, Figure 18). It produces the intended flat adapter spectrum, but substantially reduces new-task success relative to standard LoRA under the same settings, despite similar training loss. We therefore did not extend this variant to the full stream.

Based on these tests, we favor KeepLoRA with a higher adapter learning rate when replay is unavailable. Nevertheless, its old-task retention remains far below ER + LoRA at the same 15k-step budget. These results highlight the importance of replay, and we recommend using old-task data whenever they are available.

We hope these experiments address your concerns about ER's replay configuration and dedicated replay-free alternatives. We would be happy to discuss any further questions or clarify the manuscript accordingly.

Thank you for the response which has addressed my concerns. I will raise my score.

Reviewer_6vjYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how continual learning can be scaled to 125 tasks for Vision-Language-Action (VLA) models. It focuses on phase-wise continual learning, where multiple new tasks are learned together rather than one at a time. The authors compare different continual learning approaches, including experience replay (ER), LoRA, model merging, and policy distillation. Their findings show that task composition and update granularity significantly affect learning and forgetting.

Critical strengths

The paper addresses an important practical problem by studying continual learning at a much larger scale of 125 tasks. The focus on phase-wise learning makes the study relevant to real-world robotic systems, where multiple new capabilities may need to be learned together.

The paper provides an interesting analysis of how task composition and update granularity affect forgetting. In particular, the observation that certain new tasks can help preserve previously learned capabilities provides useful insights into continual learning.

The paper includes extensive comparisons with different continual learning approaches, including LoRA, model merging, and policy distillation.

Critical weaknesses

The paper mainly evaluates existing continual learning techniques rather than proposing a new method. While the large-scale evaluation is valuable, the main recommendation of using full-model experience replay is relatively straightforward.

The paper shows that certain tasks can help preserve previously learned capabilities, but it does not provide a reliable way to identify these support tasks in advance. Although several similarity-based measures are investigated, none consistently predicts which tasks will improve retention. This limits the practical usefulness of the task-composition analysis.

Questions for authors

Please refer to the weaknesses listed above.

3 comments

We thank the reviewer for recognizing the practical relevance of studying phase-wise VLA continual learning at the scale of 125 tasks, as well as the value of our task-composition analysis and extensive comparisons. We appreciate your suggestions for making the contribution and the practical use of support tasks clearer.

Note: Changes made specifically in response to Reviewer 6vjY are highlighted in gold (#946018; RGB: 148, 96, 24).


Q1: The paper mainly evaluates existing continual learning techniques rather than proposing a new method.

A: Thank you for this comment. We appreciate the concern about the contribution of a simple recipe. Our study asks which continual-learning approach works when a robot policy must absorb groups of new capabilities at the scale of 125 tasks.

Replay combined with low-rank LoRA was already a major recipe in prior VLA continual learning. For example, PHASER uses rank-32 LoRA across its backbones alongside replay, and ExpReS-VLA combines rank-32 LoRA with experience replay and retrieval. Our experiments show why this established choice needs to be revisited: LoRA helps retain old skills in small task-wise updates, but limits new-task acquisition when many tasks arrive together. Higher ranks and tuned learning rates improve acquisition, yet LoRA still forgets more intermediate-phase skills. The change in setting therefore changes which recipe works best.

Through comparisons with LoRA, model merging, reinforcement learning, and policy distillation, we find that full-model ER provides a strong practical baseline using only 2% of old-task data, without additional teacher training or online rollouts. Its new-task performance approaches that of experts trained separately from the base model, while after Phase4 its old-task success has dropped by only 0.6 percentage points on average from when those tasks were learned (Table 2). It is therefore already good enough at both acquiring new tasks and retaining old skills to serve as a strong practical baseline.

Beyond the ER recommendation, the paper offers insights into how task composition and update granularity shape acquisition and retention, and how LoRA's benefits change with task scale. These findings show that the balance between learning and forgetting changes with the task stream itself. Understanding this change is a contribution in its own right, beyond identifying a simple effective recipe.

Q2: There is not a reliable way to identify support tasks in advance.

A: Thank you for this suggestion. We have added a low-cost screening procedure with rollout validation, and further analyzed which similarities between tasks are informative about support effects (Section 4.3, Figure 5, and Appendix B.3).

Short updates identify promising support tasks before committing to longer training. We organize ten Phase2 candidates into five fixed pairs. Each candidate is briefly trained together with each of the four pairs that do not contain it, starting from the same Phase1 checkpoint and using only 64 updates per run. We rank candidates by old-task validation loss within each pair, then average their ranks across pairs. Here, a pair provides the same two companion tasks for all candidates compared within that group. Repeating the comparison across pairs accounts for the mixture dependence observed in the paper, rather than relying on a candidate's single-task retention.

The screen selects OrganizeCondiments and KettleBoiling as the top two, and PrepareToast and PrepForSanitizing as the bottom two. To test whether this short-run ranking predicts retention after longer training, we train each selected candidate for 2k updates with the same pair, RinseSinkBasin and StrainerSetup. This pair contains none of the four selected candidates. The resulting Phase1 success rates are:

Screening rankCandidatePhase1 success
1OrganizeCondiments38.6%
2KettleBoiling37.4%
9PrepareToast27.8%
10PrepForSanitizing26.5%

After only 2k training steps, the top-two mean is 38.0%, compared with 27.2% for the bottom two—already a 10.8 percentage-point retention gap under the same task count, companion tasks, and training budget. The selected top and bottom pairs also remain unchanged when any one screening pair is omitted.

The similarity analysis reveals two insights about support effects.

  1. Shared demonstration segments give a clearer signal than whole-task averages. Tasks can differ overall while sharing particular behaviors. Comparing their most similar segments focuses on these shared behaviors: for each segment, we average its highest 5% of cross-task similarities and combine both matching directions. With the same multimodal features, this gives much better agreement with the loss ranking than whole-task averages.

  2. The informative similarity involves language and actions as well as vision. VLA representations conditioned on images, instructions, and demonstration actions agree more closely with the loss ranking than vision-only features, even under the same segment-based comparison. This suggests that the relevant similarity concerns the instructed behavior as well as visual appearance.

Gradient similarity and the revised vision–language–action feature similarity both achieve Spearman correlations above 0.6 with the loss ranking; the latter also selects the same top two candidates as the loss screen. This provides a partial explanation of support effects: new tasks sharing relevant multimodal demonstration segments with old tasks are more likely to preserve old skills. For practical selection, we recommend the short-update loss screen across multiple fixed pairs, whose selected candidates are validated by the rollout comparison above.

We hope these responses clarify the paper's contribution and address your concern about identifying support tasks. We would be happy to discuss any further questions or concerns and to clarify or improve the manuscript accordingly.

Thank you to the authors for the rebuttal. However, my main concern regarding novelty remains, as the paper primarily evaluates existing continual learning methods without providing sufficiently new insights. Therefore, I will maintain my original rating.

Comments on the paper

1 comment

We thank the reviewers for their time and constructive feedback, and the AC, SAC, and PCs for coordinating the review.

We have uploaded the revised manuscript and provided point-by-point responses to every reviewer. All additions and clarifications below are incorporated into the revised manuscript.

1. Clarifying the Contribution (Reviewer 6vjY)

We clarified how phase-wise learning changes the conclusions drawn from small-scale task-wise studies, including LoRA’s advantages and the effects of task composition and update granularity. Full-model ER approaches separately trained experts on new tasks while averaging only 0.6 percentage points of old-task forgetting even after the final phase, providing a strong practical baseline for both acquisition and retention.

2. Identifying and Explaining Support Tasks (Reviewer 6vjY)

We improved our short-training tests and semantic similarity analyses. The 64-step loss screen now distinguishes stronger from weaker support-task candidates, validated by a 10.8 percentage-point retention gap after only 2k training steps between the top- and bottom-ranked candidates. The revised gradient and multimodal VLA similarity scores agree closely with the short-training loss ranking, both achieving Spearman correlations above 0.6; the multimodal score also selects the same top two candidates (Section 4.3, Figure 5, Appendix B.3).

3. Replay Sensitivity (Reviewer BysS)

The new sweeps show that replay ratio matters substantially more than memory capacity, with ratios around 30–50% providing a good acquisition–retention balance and even 0.5% memory preserving substantial old-task performance (Figure 9). We also clarified that storing a fixed fraction of each old task’s data produces a growing replay pool, while the main experiment’s strong retention supports prioritizing acquisition rather than continually increasing the replay ratio.

4. Dedicated Replay-Free Comparisons (Reviewer BysS)

We added adapted and tuned O-LoRA and KeepLoRA comparisons, with diagnostics confirming that their intended constraints operate (Appendix B.10). KeepLoRA with a higher learning rate achieves the strongest acquisition among our tested replay-free configurations, but its retention remains substantially below replay-based learning, reinforcing the importance of using old-task data whenever available.

5. ER Continuation Robustness and LoRA Analysis (Reviewer diAW)

To test whether ER continuation gains are repeatable, we added two repetitions of the final ER+3k continuation. All three runs improve Phase4 and overall success, with additional results generally within the existing 95% evaluation intervals (Appendix B.12).

We also measured high-rank LoRA’s training time and memory use: it takes longer per step and does not reduce peak device memory despite saving optimizer-state memory (Appendix B.7). To explain why high-rank LoRA still forgets more intermediate-phase skills, our spectral and perturbation analyses show that it concentrates acquired changes in a few directions, later updates interfere more strongly with those directions, and the acquired skills are more sensitive to perturbations there (Figure 8, Appendix B.9).

We hope these additions address the reviewers’ concerns and clarify the paper’s empirical contributions. As the discussion deadline approaches, we would appreciate the reviewers’ feedback on any remaining questions as soon as possible. We are eager to engage in further discussion and are fully prepared to make any additional modifications or conduct further experiments if the reviewers deem them necessary.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.