- Rating
- 3
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies continual learning of a pretrained VLA (\pi 0.5) when each update adds a batch of tasks (phase-wise) rather than one, on LIBERO and RoboCasa365.
Starting from full-model experience replay (ER) with 2% of old-task data, it compares LoRA, model merging, RETAIN, RL and on-policy distillation, with 3 findings:
(1) Task-wise conclusions do not transfer: adding LoRA to ER, a common default, mostly limits acquisition, and tuned high-rank LoRA preserves the initial tasks but forgets later-acquired skills (Phase 2: 34.7% → 27.7%, versus 35.4% → 35.3% for ER). (2) At a fixed budget, larger task groups improve both acquisition and retention, and retention depends strongly on which tasks are grouped. (3) Plain full-model ER, optionally with a short low-learning-rate continuation, approaches separately trained phase teachers without extra teachers or online rollouts.
Critical strengths
-
Per-phase reporting at meaningful scale reveals a non-obvious effect: full-model ER keeps skills learned in later phases, while tuned LoRA matches it on the initial tasks but loses them (Phase 2 after the final update: 35.3% vs 27.7%; Table 2). So an aggregate forgetting score, dominated by the 65 initial tasks, would hide this.
-
The comparison is broad and fair. LoRA is swept over rank and learning rate, and ER is compared against merging, RETAIN, RL and distillation, alongside controlled task-composition and update-granularity experiments. This makes full-model ER a credible reference baseline for future phase-wise VLA work.
Critical weaknesses
All large-scale results come from one training seed, one backbone and one task order, and the reported intervals capture only evaluation noise. Large effects, such as LoRA’s intermediate-phase forgetting, look robust, but smaller ones do not: the recommended low-learning-rate continuation (ER+3k) beats plain ER by only 1–2 points, about one interval half-width.
In addition, the paper offers no definitive explanation of why full fine-tuning with ER leads to better results. Does full fine-tuning learn something more generalizable, or does it simply produce sparser updates, which LoRA disrupts? I would like to see some reasons behind this behavior.
As for the claim that high-rank LoRA approaches full fine-tuning’s training compute, FLOPs are not the most relevant cost. LoRA mainly saves gradient and optimizer-state memory and reduces gradient communication across GPUs, which can lower hardware requirements and wall-clock time for a model of this size. Measured memory and training time would make the comparison more convincing.
Questions for authors
Do you have some deeper insights regarding the full parameter ER generalizing better? Or is it task/order specific?
Are measured peak device memory and wall-clock times available for the main full-model ER and LoRA 5× runs? If not, will you limit the efficiency comparison to the reported FLOP estimates?
2 comments
Q3: Why does full-model ER perform better than LoRA?
A: Thank you for this constructive question. We have added spectral and interference analyses to the main text and Appendix B.9 (Figures 8 and 17). Taking Phase2 as a case study, we decompose checkpoint updates by SVD, examine where the next update places its energy, and test sensitivity to perturbations. The results identify two connected reasons for LoRA's greater intermediate-phase forgetting.
First, LoRA concentrates the changes supporting acquired skills in relatively few directions. Our main LoRA configuration uses rank 128 in the language model and 256 in the action expert, yet the action expert's key/value updates need a median of only 17 singular directions to capture 95% of the update energy, compared with 99 for full-model ER (Figure 8a).
Second, subsequent LoRA learning places more energy in the directions supporting earlier skills. The cumulative Phase2-to-Phase4 LoRA update is also more concentrated and has greater total energy. On the same leading directions of the preceding LoRA update, the next actual LoRA update deposits 6.1× as much energy as the full-model ER update. The KeepLoRA analysis provides complementary evidence: at history threshold 0.99, the Phase2 historical directions alone remove 85.1% of the initial Phase3 gradient energy. Much of the incoming learning signal therefore overlaps the space associated with previous learning.
We then test whether interference in these directions has a larger functional effect. We select the leading directions of the preceding LoRA update within KeepLoRA's principal subspace and compare them with matched directions in the residual subspace. Both receiving policies use the same LoRA-defined directions. In the fixed-input test, perturbations are constructed from the actual next-phase LoRA update, with Frobenius norms matched across subspaces to keep perturbation strength equal. Adding the principal-direction perturbation produces a larger prediction-loss increase for LoRA than for full-model ER. In the rollout test, equal-norm principal-direction perturbations reduce Phase2 success by 6.3 percentage points for LoRA and 0.3 for full-model ER; residual-direction perturbations have little effect on either policy. Perturbations are added to the Phase2 weights in both tests.
Together, the concentration of acquired changes, the stronger later updates along those directions, and their greater sensitivity to interference explain why LoRA loses more intermediate-phase skills in this stream. This closely matches the mechanism reported in Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation (EBLoRA), which studies concentrated updates and their vulnerability in llms.
We also test EBLoRA's energy-balancing factorization during Phase2 learning (Appendix B.9, Figure 18). It produces the intended flat adapter spectrum, but new-task success falls substantially despite training loss similar to standard LoRA. Balancing the spectrum while preserving acquisition therefore requires further study. Together with the measured training costs, these results favor full-model ER over the tested LoRA configurations in our phase-wise setting.
The broader explanation is the change in learning dynamics. In our task-wise comparisons, new-task acquisition is relatively easy and forgetting is the dominant difficulty. With phase-wise updates, learning many new tasks together makes acquisition harder, while full-model ER already preserves old skills well. LoRA's restricted acquisition and concentrated, mutually interfering updates then become more consequential. This protocol dependence appears on both LIBERO and RoboCasa365. The detailed spectral and perturbation analyses explain the RoboCasa365 stream under the task order studied here; they do not establish task-order independence.
We hope these additions address your concerns about continuation robustness, measured efficiency, and the mechanisms behind full-model ER's advantage. We would be happy to discuss further questions and clarify the manuscript accordingly.
We thank the reviewer for recognizing that our phase-wise reporting reveals intermediate-phase forgetting that aggregate metrics can hide, and for acknowledging the breadth and fairness of our comparisons. We appreciate your questions about the repeatability of continuation gains, measured efficiency, and the reasons for full-model ER's stronger performance.
Note: Changes made specifically in response to Reviewer diAW are highlighted in soft purple (
#9370DB; RGB: 147, 112, 219).Q1: The experiments use one training seed, and the small gains from ER continuation may not be robust.
A: Thank you for raising this concern. Computational constraints limit the main four-phase experiments to a single training seed. To examine the low-learning-rate continuation gain raised in your comment, we repeat the final 3k continuation of ER+3k with seeds 43 and 44. All three runs start from the same checkpoint in the ER+3k stream, after the 15k Phase4 update and before its final 3k continuation, sharing the earlier Phase2 and Phase3 continuations. These results are now included in Appendix B.12, Table 18.
Both additional seeds improve Phase4 success and average success across all 125 tasks. Phase4 success rises from 7.9% to 9.0% and 9.3%, gains of 1.1 and 1.4 percentage points. Both runs also improve Phase2 and Phase3 success. The ER+3k result from Table 2 is included below.
The additional results are generally within the 95% evaluation intervals of ER+3k in Table 2. Although single-seed results can vary, all three continuation runs consistently improve Phase4 success and average success across all 125 tasks.
Q2: What are LoRA's measured training time and peak device memory costs?
A: Thank you for this suggestion. We have added measured step times and peak memory in Appendix B.7, Table 12. In a controlled benchmark with global batch size 512 and the main experiments' LoRA ranks of 128/256, high-rank LoRA takes longer per step than full-model ER and does not reduce peak device memory.
LoRA does save optimizer-state memory, as the reviewer notes. However, the allocator peak falls only modestly, and the measured device peak is higher. A second benchmark at 16 samples per device gives the same ordering: micro-step wall-clock time is 2.4 versus 3.3 seconds, and device peak memory is 16.7 versus 20.7 GB, for full-model ER and LoRA respectively.
The compiled LoRA step also contains more all-gathers (63→96), reduce-scatters (16→20), and all-to-alls (74→98), while the arithmetic savings at these ranks are modest. Lower-rank LoRA may offer larger efficiency gains, but its acquisition is substantially weaker in our rank ablation. The high-rank configuration that improves acquisition therefore does not retain a clear efficiency advantage in our benchmark.