- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes VLA-Collect, an interactive few-shot VLA adaptation framework. It trains multiple experts with parameter-discrepancy regularization and persistent visual transformations, uses task-completion feedback to filter and rank their rollouts, and distills the curated trajectories into a single deployment policy. Experiments on LIBERO, LIBERO-Plus, selected RoboTwin tasks, and five real-world tasks show improved performance over standard fine-tuning and single-policy self-training baselines.
Critical strengths
-
The paper presents a coherent pipeline that combines diverse rollout generation, successful-trajectory curation, and student distillation. The final policy retains the benefits of collective training without requiring multiple experts at deployment.
-
The evaluation includes matched-interaction controls, component ablations, zero-shot transfer, multiple VLA backbones, and real-robot experiments. In particular, VLA-Collect improves over matched-budget single-policy self-training on LIBERO.
Critical weaknesses
-
The few-shot framing understates the additional interaction and training cost. On LIBERO, the method uses 100 additional rollouts per task, binary completion feedback, five full expert policies, multiple training rounds, and student distillation. Comparisons with methods using only five demonstrations are therefore not directly fair, and the paper does not compare against collecting additional expert demonstrations or report the complete interaction and training cost.
-
WeightCoAct maximizes unnormalized squared distances between action-head parameters. This objective may increase parameter norms without producing meaningful behavioral diversity, and neural-network parameter distance is not functionally identifiable. The paper does not report the regularization coefficient, norm behavior, sensitivity analysis, or direct evidence that WeightCoAct increases rollout diversity.
Questions for authors
-
Please report the complete interaction, annotation, and training cost of VLA-Collect, and compare it with collecting additional expert demonstrations or another interactive baseline under a matched human-time or total-cost budget.
-
Please clarify the WeightCoAct coefficient and any mechanism that prevents unbounded parameter growth. A sensitivity analysis, action-head norm statistics, and behavior-level diversity measurements with and without WeightCoAct would help establish whether the regularizer produces complementary policies rather than merely distant parameters.
4 comments
W2 & Q2: WeightCoAct formulation, sensitivity, and behavioral diversity.
We sincerely thank the reviewer for highlighting the need to relate parameter diversity to actual behavior. This concern helped us strengthen the analysis by examining parameter norms, action differences, and task success together.
(a) Distance definition and coefficient.
Equation 1 (p. 5) uses the unsquared L2 distance, divided by the square root of each tensor's parameter count and then averaged across action-head tensors, excluding normalization and bias parameters. The regularizer combines distances to a fixed anchor and between experts. Both the backbone and action head are trained, while the regularizer applies only to the action head. This scaling accounts for tensor size and does not impose a hard norm bound; the norm statistics and action measurements are evaluated together in the ablation below.
The effective WeightCoAct coefficient increases linearly from 0.05 to 0.15 over the first 300 training steps and remains at 0.15 thereafter. Appendix C.6, “WeightCoAct Configuration and Parameter Distances” (pp. 26–27), reports this schedule, the parameter distances in Table 22 (p. 26), and the three-strength ablation in Table 23 (p. 27).
Table R3. Mean action-head parameter distances.
| Metric | Mean distance |
|---|---|
| Mean distance from experts to the fixed anchor | 0.020 |
| Mean distance over distinct expert pairs | 0.018 |
(b) Three-strength ablation: parameter norms, action differences, and success.
We compare WeightCoAct disabled, the default coefficient schedule, and twice the default strength. All three configurations use five experts, enable DataCoAct, and disable CoTC, following the Table 5 ablation setting (p. 8). The remaining training, collection, retention, and student-evaluation settings are held fixed.
Table R4. Effect of WeightCoAct strength on expert parameter norms, action differences, and final-student success (Appendix C.6, Table 23, p. 27). Norms are computed over all action-head parameters and averaged across the five experts after the second and final expert-training round. Action difference is the mean pairwise L2 distance between expert actions on the same observations. Success rates evaluate the final student.
| WeightCoAct strength | Coefficient (start → final) | Student LIBERO success (%) | Mean final action-head L2 norm | Mean pairwise action difference |
|---|---|---|---|---|
| Disabled | 0 | 81.4 | 136.3 | 0.19 |
| Default | 0.05 → 0.15 | 84.2 | 137.5 | 0.33 |
| Double | 0.10 → 0.30 | 82.6 | 138.2 | 0.41 |
Both nonzero schedules use the same 300-step linear increase. Compared with disabling WeightCoAct, the default setting changes the mean final head norm from 136.3 to 137.5 (+0.9%), increases action difference from 0.19 to 0.33, and improves student success from 81.4% to 84.2% (+2.8 percentage points). Thus, the modest difference in final parameter norms accompanies an observable change in expert actions and better student performance.
Doubling the coefficient increases the final head norm to 138.2 and action difference to 0.41, but lowers success to 82.6%, 1.6 points below the default. The default strength therefore performs best among the tested settings; greater action disagreement alone does not yield better performance. Table 5 also reports a LIBERO-Plus improvement from 46.9% to 50.3% (+3.4 points) when adding WeightCoAct with DataCoAct enabled and CoTC disabled.
(c) Qualitative rollout visualizations.
The existing rollout visualizations provide qualitative evidence of behavioral variation in the full CoAct expert pool. Figure 5 (p. 9) shows plate pushing: the single policy repeats similar approach directions and contact regions, while CoAct experts follow more varied paths. Figure 7 (p. 29) provides additional examples for drawer opening and pot grasping, with scene, object layout, initialization, instruction, and rollout budget fixed across methods.
We sincerely thank the reviewer again for the thoughtful critique and constructive suggestions. This feedback has helped us present the interaction and training costs more clearly and examine WeightCoAct more carefully, making our manuscript more precise and better supported by evidence.
The response substantially addresses my concerns through matched-interaction and matched-training baselines, as well as additional analyses of WeightCoAct’s parameter norms, behavioral diversity, and coefficient sensitivity. Although VLA-Collect remains considerably more expensive to train, the new evidence supports the effectiveness of its collective generation mechanism. I therefore raise my rating to 3.
Thank you very much for your encouraging follow-up and for taking the time to reconsider your rating. We sincerely appreciate your recognition of our efforts and are very grateful that you find our additional experiments and clarifications helpful. Your constructive feedback has been invaluable in improving both the completeness and the presentation of our work.
We sincerely thank the reviewer for the careful evaluation and constructive feedback. These comments have helped us clarify the scope of our claims, strengthen the analysis of WeightCoAct, and improve the overall quality of our manuscript. We also appreciate the reviewer's recognition of the combination of collective generation, trajectory curation, and student distillation. We address the two main concerns below: fair comparison under additional interaction and training costs, and whether WeightCoAct produces useful behavioral differences.
W1 & Q1: Interaction costs and fair comparisons.
(a) Few-shot setting and attribution of gains.
“Few-shot” refers to five initial expert demonstrations per LIBERO task. CoAct additionally collects 100 candidate rollouts per task (4,000 across 40 tasks), shared across five experts, using binary completion feedback. Initial Qwen-OFT SFT, ICL, and PEFT use no additional training interactions.
Section 4.1.1 and Table 1 (p. 6), and Appendix A.1 (p. 16), now distinguish the full adaptation gain, 75.2% → 88.9% (+13.7 percentage points), from the gain over SFT with rollouts from one policy, 81.4% → 88.9% (+7.5 points). The former includes additional interactions; the latter supports collective generation under the same interaction and retention budgets and trajectory curation (CoTC).
(b) Training costs and matched optimization budgets.
We extend SFT with rollouts from one policy to match CoAct's aggregate optimizer updates across expert and final-policy training, with the same global batch size, 100 candidate rollouts per task, CoTC, and retention schedule. This adds only 0.6 percentage points (Table R1).
Table R1. LIBERO success and recorded training + collection costs on four A800 80GB GPUs. Matched budgets use the same aggregate optimizer updates and global batch size. All methods use CoTC.
Recorded training time is 7.6 h for the original control and 22.8 h for VLA-Collect; collection adds 2.0 h per method (two phases, each collecting 2,000 trajectories in approximately one hour). Both use 5,000 updates per generation policy in each of two expert-training rounds and 10,000 final-policy updates; the first round includes initial fine-tuning. Training uses global batch size 128. Expert-training time sums the five experts' four-GPU jobs; allocated GPU-hours equal four times elapsed time. The extended control uses CoAct's total update budget. Costs cover training and collection only (Appendix B.9, Table 16, p. 23); the matched-optimization protocol is in Appendix A.3 (Table 9, p. 18).
Deployment uses one student, achieving 88.9% versus 85.1% for action ensembling of the same five experts (Appendix C.3, Table 19, p. 24).
LIBERO rollout collection and binary feedback are automated, requiring no additional human time or expert action annotation. For real-world tasks, VLA-Retrieval and CoAct each use ten initial demonstrations and ten additional physical trajectories per task; CoAct shares the latter across experts and uses human completion judgments (Appendix B.3, p. 20).
(c) Interactive baselines: SFT on collected rollouts and GRPO.
The SFT baseline collects rollouts, selects them using completion feedback and CoTC, and retrains on retained trajectories plus initial demonstrations. We also evaluate Qwen-OFT + GRPO from a five-shot SFT checkpoint with binary task-completion rewards (Appendix A.2, Table 8, p. 17).
Table R2. LIBERO interactive baselines, all using five initial demonstrations per task. The SFT controls use CoTC; extended training matches CoAct's aggregate updates and global batch size.
VLA-Collect exceeds the matched-training SFT control by 6.9 points and GRPO by 7.3 points. The first comparison controls for additional optimization; the second compares against reward-based policy improvement under a similar interaction budget. GRPO trains one checkpoint per suite; CoAct produces one student jointly trained across all four suites.