- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes KEEP, a dual-teacher distillation framework for adapting flow-based Vision-Language-Action policies while preserving generalist capabilities. A competence gate compares the flow-matching errors of a LoRA expert and a frozen generalist, guiding skill injection and knowledge anchoring. Extensive real-robot, REALM simulation, and LIBERO experiments demonstrate improved acquisition–retention trade-offs.
Critical strengths
KEEP introduces a sample-dependent dual-teacher distillation mechanism that dynamically balances novel-skill acquisition and generalist knowledge preservation, offering a meaningful alternative to global weight-space merging. The paper provides extensive empirical validation, including 5,150 real-robot rollouts and over 250,000 simulated episodes. KEEP achieves 81% novel-task success and 59% generalist success, outperforming the strongest merging baseline (57% and 46%). The additional LIBERO evaluations and systematic ablations further strengthen the experimental evidence.
Critical weaknesses
W1. KEEP requires a separately trained LoRA expert and access to replayed pre-training data for each new skill, limiting applicability when historical data are unavailable and increasing adaptation costs.
W2. Limited validation of long-term continual learning. The evaluation covers at most two new skills, and Figure 3 shows substantial forgetting without earlier-task replay, leaving long-term knowledge preservation and scalability to longer task sequences unestablished.
W3. Although flow-matching error is mathematically linked to demonstration-action prediction error, the paper does not directly validate whether it reliably predicts teachers' closed-loop task competence, particularly on student-generated trajectories.
W4. Confounded ablation of anchoring and replay. In Figure 4, removing the anchor term also removes replay data, confounding their contributions and preventing an isolated assessment of the anchoring objective under identical replay settings.
W5. Although Appendix A.7 reports substantially higher training costs for KEEP, the paper lacks acquisition–retention comparisons under matched end-to-end GPU-hour budgets, including expert training and student rollout costs.
Questions for authors
Please refer to Weaknesses W1–W5 for my questions and concerns.
4 comments
W3: Does the flow-matching error predict the teachers’ task competence, particularly on student-generated trajectories?
The competence gate addresses a narrower question than task competence. On each training sample it compares the two teachers and selects the one whose predicted action is closer to the demonstrated action, i.e., the one with the lower flow-matching error. It does not predict how well a teacher performs the task, and it does not use any rollouts. Section 3 now explicitly describes the assumption behind the competence gate.
Whether this comparison selects the right teacher is tested in Table 1. There we replace the competence gate by an oracle that knows the source of each sample and assigns every new-skill sample to the LoRA expert and every replayed sample to the base policy. The competence gate is as good as or better than this oracle (row "Task label" in Table 1) on all three measures: 71.1% vs. 68.2% on pouring, 91.6% vs. 87.4% on stacking, and 59.8% vs. 59.1% on the generalist suite. This shows that comparing flow-matching errors selects the teacher at least as well as knowing the source of each sample.
Regarding student-generated trajectories, the gate is always computed on demonstrations, where the target is known, and not on the student’s own actions. The student’s own actions serve only as the input at which the expert is queried for the inject term, which raises novel-task success from 71.1% to 81.3% without a measurable cost to generalist success (Figure 5).
W4: The ablation of the anchor also removed the replay data
You are right, thank you for pointing this out. We have rerun the ablation. The runs without the anchor term now use the same 80/20 mixture as KEEP. Generalist success is 20.1–21.0% without the anchor term, compared to 59.8% with it on identical data, and novel-task success is 61.1–61.9% without it, compared to 81.3% with it. This shows that the retention comes from the anchor term, and is not provided by the replay data alone. These numbers are now reported in Figure 5 and Section 4.3 of the revised paper (Figure 4 in the original submission) and replace the previously reported numbers.
W5: No comparison under matched end-to-end GPU-hours
The cost is now in the main text (Section 4): about 45 GPU-hours for KEEP, including the training of its LoRA expert and the student’s own ODE integration, compared to about 17 for co-training followed by merging. We also trained R1 baselines, FFT + RETAIN and Co-Train + RETAIN, at a matched budget of 45 GPU-hours (13,000 steps under our recipe R1; denoted RETAIN-L in Appendix A.6 and Figures 16 and 17). The results in simulation after merging are given in the table below. Neither matched-budget run closes the gap on the novel tasks. FFT + RETAIN also retains much less generalist success than KEEP. Co-Train + RETAIN retains 60.7%, compared to 59.8% for KEEP. Please note that simulation overestimates the generalist success of the co-trained baselines by 14 to 25 points relative to the real robot, but not that of KEEP (Appendix A.5). Independently of these runs, continuing the stronger RETAIN recipe (R2) beyond its stopping point lowers both success rates (Appendix A.6, Figure 17), so the gap between the baselines and KEEP on the novel tasks does not come from a smaller training budget.
| Method (45 GPU-hours) | Pouring | Stacking | Generalist |
|---|---|---|---|
| FFT + RETAIN-L | 37.8 | 80.6 | 32.3 |
| Co-Train + RETAIN-L | 24.7 | 73.8 | 60.7 |
| KEEP | 71.1 | 91.6 | 59.8 |
Success rate [%] in simulation. The generalist column is the mean over the two single-task models, as in Table 1.
[1] Liu et al., Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning, ICML 2026
Thank you for the additional experiments and detailed cost analysis. I acknowledge that the matched-budget comparisons strengthen the empirical evidence for KEEP. However, my primary concern regarding its practical adaptation cost and scalability remains unresolved. Each new skill still requires training a separate LoRA expert followed by costly student distillation, while preserving previously acquired skills continues to depend on historical data replay.
Recent VLA continual learning methods, such as CLARE (IEEE ROBOTICS AND AUTOMATION LETTERS, 2026), have explored replay-free adaptation. They demonstrate that these dependencies are not universal requirements of continual VLA learning.
Given the importance on efficiency and scalability, I decide to maintain my original rating.
Thank you for the reply, and for acknowledging that the matched-budget comparison strengthens the evidence. We would like to clarify how CLARE relates to our setting, which we also discuss in Section 2 and Appendix A.6.
CLARE shows that sequentially added tasks can be retained without replay data. It does not evaluate the question our paper addresses: retention of the pre-trained policy's own abilities. In both its simulation and real-robot experiments, the policy is evaluated only on the tasks added during continual learning, and no result is reported on the original tasks of the pre-trained policy. In our paper this is the central measure: the base policy reaches 61% on our generalist suite, and KEEP stays within noise of it. CLARE's router always activates an adapter in each adapted layer, so the base policy is never run unmodified, and there is no route back to it. Whether a routing approach can retain the base policy's abilities without access to the pre-training distribution is therefore untested.
CLARE is also demonstrated at a different scale. In simulation, its base policies are compact models of about 200M parameters pre-trained by the authors on LIBERO-90, not a large pre-trained VLA such as . Our results are on with 5,150 real-robot rollouts, and in this setting RETAIN is the retention method previously demonstrated, which we compare against in every experiment, including at a matched budget. KEEP also produces a single network with the size and inference cost of the base policy, whereas CLARE adds parameters with every task.
We agree that replay-free retention is an important goal, and the paper states the need for replay data and a LoRA expert as limitations, as well as the decline in novel-task success over longer skill sequences. For a single fixed-size network that keeps the abilities of a large pre-trained VLA, we believe replay-free retention remains an open problem, and we will expand the discussion of CLARE in the paper accordingly.
Thank you for the detailed review. Three of the weaknesses led to new experiments (W2, W4 and W5), and the others to clarifications in the paper. We answer them in order.
W1: KEEP needs a LoRA expert and replayed pre-training data
Both are indeed properties of the method, and the paper states them as limitations. However, we would like to note two points. First, the replay data is the same data that the co-training baselines use, i.e., a uniform subset of about 5% of DROID. Second, training the LoRA expert accounts for 1.9 of the 5.6 hours that adding a skill takes. The ablation of the co-training ratio (Figure 5, Section 4.3) also shows that the replay data is needed for the anchor term to act on.
Retaining prior skills without any of the original training data is, to our knowledge, still an open problem. The methods we evaluate that do not use replay data, FFT + RETAIN and the FFT models trained with a weight penalty, do not reach KEEP’s combination of novel-task and generalist success (Figure 2, Appendix A.6). In its current form, KEEP uses the replayed actions to compute the competence gate and to form the noised action at which the anchor term is evaluated. Both uses could in principle be replaced: assigning every replayed sample to the base policy performs similarly to the gate (Table 1), and the anchor term can be evaluated on the student’s own actions (Figure 5). Investigating the method without access to the original data is therefore an interesting direction for future work, and we expect world models, as a source of such data, to play a useful role there.
W2: At most two new skills, and long-term preservation is not established
We extended the evaluation to five skills. On LIBERO, we add them one at a time, in three task orders, and re-evaluate the generalist suite and every learned skill after each stage (new Section 4.2, Figure 4, Appendix A.2). KEEP retains generalist success and outperforms the baseline on the novel tasks also in this five-skill setting:
Earlier skills are retained by replaying their demonstrations at every stage. This is the mechanism shown in Figure 3, and [1] found the same for pre-trained VLAs on LIBERO. Since both methods replay the earlier skills, the advantage of KEEP lies in how well each new skill is acquired, which is most visible on the harder splits. However, retention is harder on the real robot than on LIBERO. There, Co-Train + RETAIN, which uses the same replay data as KEEP, loses about twenty points of generalist success compared to the base policy (61%), averaged over the three tasks. KEEP stays within three points of the base policy (Figure 2). In other words, the advantage of KEEP in retaining the original tasks is larger on the real robot than on LIBERO.