acceptodds
Under review as a conference paper at ICLR 2027

VLA-Collect: Training Vision Language Action Models with Few-shot Data via Collective Action Fine-Tuning

Abstract

Vision-Language-Action (VLA) models unify perception, language, and action and achieve strong performance across robotic tasks, but high-fidelity dexterous manipulation often still relies on costly supervised fine-tuning (SFT), pushing deployment into a few-shot regime. With few demonstrations, a policy can overfit to narrow behaviors and encounter unfamiliar states during execution. When self-generated rollouts are reused for training, low-quality or repetitive trajectories can reinforce execution errors and limited coverage. We propose Collective Action Fine-Tuning (CoAct), a collective-learning paradigm for interactive few-shot VLA post-training using limited initial expert demonstrations, additional rollouts, and binary task-completion feedback. CoAct maintains a diverse policy pool with two mechanisms: WeightCoAct encourages action-head discrepancy, while DataCoAct assigns persistent expert-specific visual views to broaden the experience generated by the pool. CoAct further introduces Collective Trajectory Curation (CoTC) to select successful rollouts and favor experience beyond the initial demonstration coverage, then distills curated trajectories into a unified student policy. Experiments on LIBERO, LIBERO-Plus, and RoboTwin, complemented by five real-world tasks, show improved few-shot performance and zero-shot transfer.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Collective learning, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_yTsWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VLA-Collect, an interactive few-shot VLA adaptation framework. It trains multiple experts with parameter-discrepancy regularization and persistent visual transformations, uses task-completion feedback to filter and rank their rollouts, and distills the curated trajectories into a single deployment policy. Experiments on LIBERO, LIBERO-Plus, selected RoboTwin tasks, and five real-world tasks show improved performance over standard fine-tuning and single-policy self-training baselines.

Critical strengths
  1. The paper presents a coherent pipeline that combines diverse rollout generation, successful-trajectory curation, and student distillation. The final policy retains the benefits of collective training without requiring multiple experts at deployment.

  2. The evaluation includes matched-interaction controls, component ablations, zero-shot transfer, multiple VLA backbones, and real-robot experiments. In particular, VLA-Collect improves over matched-budget single-policy self-training on LIBERO.

Critical weaknesses
  1. The few-shot framing understates the additional interaction and training cost. On LIBERO, the method uses 100 additional rollouts per task, binary completion feedback, five full expert policies, multiple training rounds, and student distillation. Comparisons with methods using only five demonstrations are therefore not directly fair, and the paper does not compare against collecting additional expert demonstrations or report the complete interaction and training cost.

  2. WeightCoAct maximizes unnormalized squared distances between action-head parameters. This objective may increase parameter norms without producing meaningful behavioral diversity, and neural-network parameter distance is not functionally identifiable. The paper does not report the regularization coefficient, norm behavior, sensitivity analysis, or direct evidence that WeightCoAct increases rollout diversity.

Questions for authors
  1. Please report the complete interaction, annotation, and training cost of VLA-Collect, and compare it with collecting additional expert demonstrations or another interactive baseline under a matched human-time or total-cost budget.

  2. Please clarify the WeightCoAct coefficient and any mechanism that prevents unbounded parameter growth. A sensitivity analysis, action-head norm statistics, and behavior-level diversity measurements with and without WeightCoAct would help establish whether the regularizer produces complementary policies rather than merely distant parameters.

4 comments

We sincerely thank the reviewer for the careful evaluation and constructive feedback. These comments have helped us clarify the scope of our claims, strengthen the analysis of WeightCoAct, and improve the overall quality of our manuscript. We also appreciate the reviewer's recognition of the combination of collective generation, trajectory curation, and student distillation. We address the two main concerns below: fair comparison under additional interaction and training costs, and whether WeightCoAct produces useful behavioral differences.

W1 & Q1: Interaction costs and fair comparisons.

(a) Few-shot setting and attribution of gains.

“Few-shot” refers to five initial expert demonstrations per LIBERO task. CoAct additionally collects 100 candidate rollouts per task (4,000 across 40 tasks), shared across five experts, using binary completion feedback. Initial Qwen-OFT SFT, ICL, and PEFT use no additional training interactions.

Section 4.1.1 and Table 1 (p. 6), and Appendix A.1 (p. 16), now distinguish the full adaptation gain, 75.2% → 88.9% (+13.7 percentage points), from the gain over SFT with rollouts from one policy, 81.4% → 88.9% (+7.5 points). The former includes additional interactions; the latter supports collective generation under the same interaction and retention budgets and trajectory curation (CoTC).

(b) Training costs and matched optimization budgets.

We extend SFT with rollouts from one policy to match CoAct's aggregate optimizer updates across expert and final-policy training, with the same global batch size, 100 candidate rollouts per task, CoTC, and retention schedule. This adds only 0.6 percentage points (Table R1).

Table R1. LIBERO success and recorded training + collection costs on four A800 80GB GPUs. Matched budgets use the same aggregate optimizer updates and global batch size. All methods use CoTC.

MethodTraining budgetTraining + collection time (h)Allocated GPU-hoursLIBERO success (%)
SFT + rollouts from one policyOriginal training budget9.638.481.4
SFT + rollouts from one policyMatched training budget24.899.282.0
VLA-CollectMatched training budget24.899.288.9

Recorded training time is 7.6 h for the original control and 22.8 h for VLA-Collect; collection adds 2.0 h per method (two phases, each collecting 2,000 trajectories in approximately one hour). Both use 5,000 updates per generation policy in each of two expert-training rounds and 10,000 final-policy updates; the first round includes initial fine-tuning. Training uses global batch size 128. Expert-training time sums the five experts' four-GPU jobs; allocated GPU-hours equal four times elapsed time. The extended control uses CoAct's total update budget. Costs cover training and collection only (Appendix B.9, Table 16, p. 23); the matched-optimization protocol is in Appendix A.3 (Table 9, p. 18).

Deployment uses one student, achieving 88.9% versus 85.1% for action ensembling of the same five experts (Appendix C.3, Table 19, p. 24).

LIBERO rollout collection and binary feedback are automated, requiring no additional human time or expert action annotation. For real-world tasks, VLA-Retrieval and CoAct each use ten initial demonstrations and ten additional physical trajectories per task; CoAct shares the latter across experts and uses human completion judgments (Appendix B.3, p. 20).

(c) Interactive baselines: SFT on collected rollouts and GRPO.

The SFT baseline collects rollouts, selects them using completion feedback and CoTC, and retrains on retained trajectories plus initial demonstrations. We also evaluate Qwen-OFT + GRPO from a five-shot SFT checkpoint with binary task-completion rewards (Appendix A.2, Table 8, p. 17).

Table R2. LIBERO interactive baselines, all using five initial demonstrations per task. The SFT controls use CoTC; extended training matches CoAct's aggregate updates and global batch size.

MethodAdditional training rollouts, totalLIBERO success (%)
SFT + rollouts from one policy4,00081.4
SFT + rollouts from one policy (extended training)4,00082.0
Qwen-OFT + GRPO4,09681.6
VLA-Collect4,00088.9

VLA-Collect exceeds the matched-training SFT control by 6.9 points and GRPO by 7.3 points. The first comparison controls for additional optimization; the second compares against reward-based policy improvement under a similar interaction budget. GRPO trains one checkpoint per suite; CoAct produces one student jointly trained across all four suites.

W2 & Q2: WeightCoAct formulation, sensitivity, and behavioral diversity.

We sincerely thank the reviewer for highlighting the need to relate parameter diversity to actual behavior. This concern helped us strengthen the analysis by examining parameter norms, action differences, and task success together.

(a) Distance definition and coefficient.

Equation 1 (p. 5) uses the unsquared L2 distance, divided by the square root of each tensor's parameter count and then averaged across action-head tensors, excluding normalization and bias parameters. The regularizer combines distances to a fixed anchor and between experts. Both the backbone and action head are trained, while the regularizer applies only to the action head. This scaling accounts for tensor size and does not impose a hard norm bound; the norm statistics and action measurements are evaluated together in the ablation below.

The effective WeightCoAct coefficient increases linearly from 0.05 to 0.15 over the first 300 training steps and remains at 0.15 thereafter. Appendix C.6, “WeightCoAct Configuration and Parameter Distances” (pp. 26–27), reports this schedule, the parameter distances in Table 22 (p. 26), and the three-strength ablation in Table 23 (p. 27).

Table R3. Mean action-head parameter distances.

MetricMean distance
Mean distance from experts to the fixed anchor0.020
Mean distance over distinct expert pairs0.018

(b) Three-strength ablation: parameter norms, action differences, and success.

We compare WeightCoAct disabled, the default coefficient schedule, and twice the default strength. All three configurations use five experts, enable DataCoAct, and disable CoTC, following the Table 5 ablation setting (p. 8). The remaining training, collection, retention, and student-evaluation settings are held fixed.

Table R4. Effect of WeightCoAct strength on expert parameter norms, action differences, and final-student success (Appendix C.6, Table 23, p. 27). Norms are computed over all action-head parameters and averaged across the five experts after the second and final expert-training round. Action difference is the mean pairwise L2 distance between expert actions on the same observations. Success rates evaluate the final student.

WeightCoAct strengthCoefficient (start → final)Student LIBERO success (%)Mean final action-head L2 normMean pairwise action difference
Disabled081.4136.30.19
Default0.05 → 0.1584.2137.50.33
Double0.10 → 0.3082.6138.20.41

Both nonzero schedules use the same 300-step linear increase. Compared with disabling WeightCoAct, the default setting changes the mean final head norm from 136.3 to 137.5 (+0.9%), increases action difference from 0.19 to 0.33, and improves student success from 81.4% to 84.2% (+2.8 percentage points). Thus, the modest difference in final parameter norms accompanies an observable change in expert actions and better student performance.

Doubling the coefficient increases the final head norm to 138.2 and action difference to 0.41, but lowers success to 82.6%, 1.6 points below the default. The default strength therefore performs best among the tested settings; greater action disagreement alone does not yield better performance. Table 5 also reports a LIBERO-Plus improvement from 46.9% to 50.3% (+3.4 points) when adding WeightCoAct with DataCoAct enabled and CoTC disabled.

(c) Qualitative rollout visualizations.

The existing rollout visualizations provide qualitative evidence of behavioral variation in the full CoAct expert pool. Figure 5 (p. 9) shows plate pushing: the single policy repeats similar approach directions and contact regions, while CoAct experts follow more varied paths. Figure 7 (p. 29) provides additional examples for drawer opening and pot grasping, with scene, object layout, initialization, instruction, and rollout budget fixed across methods.

We sincerely thank the reviewer again for the thoughtful critique and constructive suggestions. This feedback has helped us present the interaction and training costs more clearly and examine WeightCoAct more carefully, making our manuscript more precise and better supported by evidence.

The response substantially addresses my concerns through matched-interaction and matched-training baselines, as well as additional analyses of WeightCoAct’s parameter norms, behavioral diversity, and coefficient sensitivity. Although VLA-Collect remains considerably more expensive to train, the new evidence supports the effectiveness of its collective generation mechanism. I therefore raise my rating to 3.

Thank you very much for your encouraging follow-up and for taking the time to reconsider your rating. We sincerely appreciate your recognition of our efforts and are very grateful that you find our additional experiments and clarifications helpful. Your constructive feedback has been invaluable in improving both the completeness and the presentation of our work.

Reviewer_4s2POpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

VLA-Collect is about training vision-language-action models when you only have a handful of expert demos. Instead of collecting new data with a single policy, they train several expert policies and let each of them roll out different trajectories. The experts are made diverse in two ways. WeightCoAct pushes their action heads apart in parameter space, and DataCoAct gives each expert its own fixed visual setting (camera view, lighting, background, etc.). Failed trajectories get filtered out using task success labels, and the successful ones are ranked by trajectory length, gripper movement, and how visually different they look from the original demos. The selected trajectories then go into training one final policy. They evaluate on LIBERO, LIBERO-Plus, five RoboTwin tasks, and five real-world tasks. It beats the initial policy and several single-policy training methods, and the collected data also improves several other VLA models.

Critical strengths
  1. Strong results across several settings: improvements on all four LIBERO suites, better transfer to LIBERO-Plus, plus gains on RoboTwin and on a real robot. These results suggest that the method is not limited to one simulator.
  2. Good comparisons and ablations. They compare against single-policy collection with the same rollout budget, ablate WeightCoAct, DataCoAct, and trajectory selection separately, show the collected data helps six other VLA models, and show the student beats directly combining the five experts' actions.
Critical weaknesses
  1. One obvious baseline is missing: several independently trained experts without WeightCoAct or DataCoAct. All tested multi-expert settings use at least one of the proposed diversity methods, so it's hard to say whether the improvement comes from the diversity methods themselves or just from training a few policies and pooling their data. An independently trained experts baseline with the same rollout budget and the same selection step would make this clearer.
  2. The total cost and reliability are unclear. Five experts, three training rounds, 100 extra rollouts per LIBERO task, plus a final student training stage, but the paper does not report total training time or total compute. The reported uncertainty comes from re-evaluating the same trained policy, not from different training seeds or different sets of initial demos. The method is also unstable with only one demonstration. All of this makes it hard to judge what the method actually costs and how stable it is.
Questions for authors
  1. How does VLA-Collect compare to several independently trained experts (no WeightCoAct, no fixed per-expert views) with the same rollout budget and the same CoTC?
  2. What is the total training time or compute for the whole pipeline, including all experts, three rounds, rollout collection, and final student training? And how does that compare to the single-policy baseline?
  3. Are the gains stable across training seeds and across different choices of the initial demonstrations? The repeated evaluations in the appendix look like they reuse the same trained policy.
3 comments

We sincerely thank the reviewer for the careful evaluation and constructive feedback. These comments have helped us strengthen the evaluation of training reliability and improve the clarity and completeness of our manuscript. We also appreciate the reviewer's recognition of the results across simulation and real-world tasks, the comparisons under matched rollout budgets, and the transferability of the collected supervision. We address the independent-expert comparison, training cost, and reliability below.

Q1: Independent experts without WeightCoAct or DataCoAct.

We evaluated a pool of five complete VLA policies initialized from the same pretrained checkpoint and trained independently using five different training random seeds and the same initial demonstrations. Each policy updates both its backbone and action head, without WeightCoAct or expert-specific visual augmentation. The pool shared a total of 100 candidate rollouts per LIBERO task. The collection schedule, trajectory curation (CoTC) procedure, retention budget, student-training protocol, and aggregate expert-training updates were matched to CoAct. The protocol, results, and discussion have been added to Appendix A.5, “Comparison with Independently Trained Experts” (pp. 18–19, Table 11 on p. 19).

Table R1. LIBERO success with rollouts collected by one policy or five experts. All rows evaluate the final student and use the same trajectory curation.

Rollout sourcePolicies used for collectionTotal candidate rollouts per taskStudent LIBERO success (%)
Rollouts from one policy110081.4
Rollouts from multiple policies (different seeds)510082.8
Rollouts from multiple policies (CoAct)510088.9

The student trained from the independent expert pool achieves 82.8%, while VLA-Collect reaches 88.9%, a 6.1-percentage-point advantage. With the number of experts and the collection, curation, and student-training procedures controlled, this result supports the value of deliberate expert diversification beyond pooling independently trained policies.

Q2: Training cost and comparison under matched optimization budgets.

Under matched optimization budgets, VLA-Collect achieves 88.9% success versus 82.0% for extended SFT with rollouts from one policy, a 6.9-percentage-point advantage (Table R3). We have added the supplemental hardware and training configurations, cost comparison, and timing details to Appendix B.9, “Compute Resources” (p. 23, Table 16).

Table R2 reports recorded training and collection time. Training uses global batch size 128. The original single-policy control and CoAct use 5,000 updates per generation policy in each of the two expert-training rounds and 10,000 final-policy updates. The first-round budget includes initial fine-tuning. Each of the two collection phases takes approximately one hour for 2,000 trajectories (40 tasks × 50 trajectories).

Table R2. Recorded training and collection costs on four A800 80GB GPUs. Both methods use the same trajectory curation. The control uses the original training budget. Expert-training times sum the five experts' four-GPU training jobs.

StageSFT + rollouts from one policyVLA-Collect
Expert round 1: 5 demonstrations per task1.9 h9.5 h
Expert round 2: 5 demonstrations + 10 retained rollouts per task1.9 h9.5 h
Final-policy training: 5 demonstrations + 25 retained rollouts per task3.8 h3.8 h
Training subtotal7.6 h22.8 h
Collection: 4,000 candidate trajectories in total2.0 h2.0 h
Training + collection total9.6 h24.8 h
Allocated GPU-hours (4 GPUs)38.4 GPU-hours99.2 GPU-hours

The subtotal covers training and collection, with GPU-hours calculated as four GPUs × elapsed hours. The single-policy column uses the original training schedule.

We also extended SFT with rollouts from one policy to match CoAct's aggregate optimizer updates across expert and final-policy training, keeping the global batch size, 100 candidate rollouts per task, CoTC, and retention schedule fixed. Appendix A.3 reports this comparison (p. 18, Table 9).

Table R3. LIBERO success and recorded training and collection costs on four A800 80GB GPUs. Matched training budget denotes matched aggregate optimizer updates and global batch size. All methods use the same trajectory curation.

MethodTraining budgetTraining + collection time (h)Allocated GPU-hoursLIBERO success (%)
SFT + rollouts from one policyOriginal training budget9.638.481.4
SFT + rollouts from one policyMatched training budget24.899.282.0
VLA-CollectMatched training budget24.899.288.9

Longer single-policy training adds only 0.6 percentage points, supporting the benefit of collective generation under matched optimization budgets. The expert pool is used offline; deployment uses one VLA-Collect policy with the same architecture as its underlying backbone.

Q3: Training reliability and sensitivity to the initial demonstrations.

We added two experiments: varying the training seed with the same five initial demonstrations per task, and varying the five initial demonstrations with the training seed fixed. Each experiment includes three complete CoAct runs, covering two rounds of expert training (including initial fine-tuning in the first round), two rollout collection phases with CoTC, and final student training. We keep the model, hyperparameters, interaction and retention budgets, and evaluation conditions fixed, and evaluate each final student on all 40 LIBERO tasks with 50 trials per task. The protocols and individual results are reported in Appendices C.7 and C.8 (pp. 27–28; Tables 24 and 25).

Table R4. Final-student LIBERO success (%) when varying training seeds or initial demonstration sets. Each row includes three complete training runs. SD is the sample standard deviation.

Varied settingOriginal runAdditional run 1Additional run 2Mean ± SD (%)
Training random seed88.989.389.089.07 ± 0.21
Initial demonstration set (5 per task)88.988.689.188.87 ± 0.25

Both experiments include the same original run (88.9%), giving five distinct training runs in total. The small variation supports consistent performance across the tested training seeds and initial demonstration sets. Appendix C.1 (p. 23) separately reports the variability from repeated evaluations of a single trained policy.

With one initial demonstration, the policy produces too few successful trajectories under the fixed interaction budget, and CoAct remains unstable. With three demonstrations per task, VLA-Collect reaches 81.6%, compared with 68.7% for the initial SFT policy; with five demonstrations, it reaches 88.9% (Appendix C.5, p. 25; Table 21).

We sincerely thank the reviewer again for the careful assessment and helpful suggestions. These comments have guided us toward stronger comparisons and a more thorough evaluation of training reliability, improving both the rigor and completeness of our manuscript.

Reviewer_5CbtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies VLA adaptation with few initial expert demonstrations, additional environment interactions, and binary task-completion feedback. The goal is to improve policy performance without collecting further expert action annotations.

The proposed CoAct framework trains multiple complete VLA policies. WeightCoAct encourages separation between their action-head parameters, while DataCoAct assigns persistent visual transformations to individual experts. CoTC filters unsuccessful rollouts and ranks successful trajectories using gripper motion, trajectory length, and visual novelty. The retained trajectories and initial demonstrations train a single deployment policy through behavior cloning.

Experiments cover LIBERO, zero-shot transfer to LIBERO-Plus, five selected RoboTwin tasks, and five real-world tasks. On LIBERO, VLA-Collect achieves 88.9% success, compared with 81.4% for single-policy self-training with CoTC under matched rollout and retention budgets. The collected data also improves six additional VLA backbones.

Critical strengths
  • The complete pipeline substantially improves success rates. On LIBERO, CoAct outperforms single-policy self-training with the same CoTC procedure by 7.5 percentage points under matched budgets for rollout attempts and retained training trajectories. The method also improves performance on LIBERO-Plus and the evaluated RoboTwin and real-world tasks.
  • The collected data improves six additional VLA backbones on both LIBERO and LIBERO-Plus, suggesting that the improvement is not specific to one VLA architecture. The final system deploys a single student policy, avoiding the inference cost of running the expert pool.
Critical weaknesses
  1. The benefit of the expert-training machinery is not sufficiently established relative to its complexity.

CoAct trains separate VLA backbones and action heads, collects their rollouts, and then trains a student. Table 5 shows that combining WeightCoAct and DataCoAct improves performance, but the main-paper comparisons do not establish their advantage over an equally sized pool of independently trained policies without these mechanisms. The training overhead is also not quantified in the main text.

DataCoAct introduces an additional practical question. It assigns persistent visual transformations to individual experts, but Section 3.3 does not make clear how viewpoint and scene-appearance changes are generated for real-world recordings or what observations the experts receive during collection. These transformations are feasible in principle, but their implementation and the benefit of persistent expert-specific assignments over shared augmentation remain unclear.

  1. The CoTC ranking criteria are insufficiently validated. Gripper motion, trajectory length, and visual novelty are plausible selection signals, but the experiments do not establish that they identify better training examples among successful trajectories. Adding CoTC improves LIBERO success from 84.2% to 88.9% in Table 5, but this ablation does not separately report the effects of success filtering and ranking. A comparison against random selection from the same successful rollout pool would clarify whether the proposed ranking contributes beyond removing failures.
Questions for authors
  • How are DataCoAct’s viewpoint and scene-appearance transformations generated for real-world data, and what observations does each expert receive during rollout collection?
  • How does CoAct compare with an equally sized pool of independently trained policies under the same collection, curation, and student-training procedure? What is the additional training cost, and does persistent view assignment improve over giving all experts the same augmentation distribution?
  • Does the configuration without CoTC in Table 5 already exclude failed trajectories? How does full CoTC compare with random selection from the same successful rollout pool under the same retention budget?
3 comments

We sincerely thank the reviewer for the careful evaluation and constructive feedback. These comments have helped us strengthen our experimental controls and improve the clarity and rigor of our manuscript. We also appreciate the reviewer's recognition of the gains under matched interaction and retention budgets, the transferability of the collected supervision, and the use of a single student at deployment.

Q1: DataCoAct implementation for real-world data and rollout inputs.

We sincerely thank the reviewer for highlighting the need to clarify how DataCoAct constructs visual views and how these views are used during collection. In our real-world experiments, DataCoAct augments the visual observations in recorded trajectories while retaining the associated language instructions, robot states, action targets, and timestamps. Each expert is assigned a fixed augmentation family: identity, image-space framing perturbations, photometric changes, background variation, or mild sensor noise.

For framing perturbations, we use small image translations and crops that retain the task-relevant workspace. These perturbations vary image framing rather than reconstructing a new three-dimensional camera viewpoint. Photometric augmentation adjusts brightness and contrast. Background augmentation is restricted to regions outside the robot, task objects, target locations, and contact areas. Geometric and photometric parameters and the background template remain fixed within each augmented trajectory, with foreground masks updated across frames.

During rollout collection, all experts receive the original camera observations through the standard policy preprocessing, together with the robot state and task instruction. Each augmented expert is trained on a mixture of original observations and observations from its assigned augmentation family, providing training exposure to the inputs used during collection. We record the original observations and executed actions; retained trajectories are augmented when used for subsequent expert training, while the final student is trained and deployed on original observations.

Q2: Benefits and costs of expert training.

(a) Comparison with an equally sized pool of independent experts.

We sincerely thank the reviewer for suggesting this direct comparison. We evaluated a pool of five complete VLA policies initialized from the same pretrained checkpoint and trained independently using five different training random seeds and the same initial demonstrations. Each policy updates both its backbone and action head, without WeightCoAct or expert-specific visual augmentation. The pool shared a total of 100 candidate rollouts per LIBERO task. The collection schedule, trajectory curation (CoTC) procedure, retention budget, student-training protocol, and aggregate expert-training updates were matched to CoAct. The protocol, results, and discussion have been added to Appendix A.5, “Comparison with Independently Trained Experts” (pp. 18–19, Table 11 on p. 19).

Table R1. LIBERO success with rollouts collected by one policy or five experts. All rows evaluate the final student and use the same trajectory curation.

Rollout sourcePolicies used for collectionTotal candidate rollouts per taskStudent LIBERO success (%)
Rollouts from one policy110081.4
Rollouts from multiple policies (different seeds)510082.8
Rollouts from multiple policies (CoAct)510088.9

The student trained from the independent expert pool achieves 82.8%, while VLA-Collect reaches 88.9%, a 6.1-percentage-point advantage. With the number of experts and the collection, curation, and student-training procedures controlled, this result supports the value of deliberate expert diversification beyond pooling independently trained policies.

(b) Training overhead and comparison under matched optimization budgets.

We sincerely thank the reviewer for raising the training-cost concern. On four A800 80GB GPUs, recorded training time, including final-policy training, increases from 7.6 hours for the original single-policy control to 22.8 hours for VLA-Collect. Each method collects 4,000 candidate trajectories in approximately 2.0 hours, giving training and collection subtotals of 9.6 and 24.8 hours, respectively. The cost comparison, training configuration, and timing details are reported in Appendix B.9, “Compute Resources” (p. 23, Table 16).

Training uses global batch size 128. The original single-policy control and CoAct use 5,000 updates per generation policy in each of the two expert-training rounds and 10,000 final-policy updates. The first-round budget includes initial fine-tuning. Expert-training times sum the five experts' four-GPU training jobs. GPU-hours count four allocated GPUs, and the reported times cover training and collection only.

To assess whether the improvement can be explained by additional optimization, we extended SFT with CoTC-selected rollouts from one policy to match CoAct's aggregate optimizer-update budget across expert and final-policy training. We kept the global batch size, 100 candidate rollouts per task, CoTC procedure, and retention schedule unchanged. The protocol, results, and discussion are included in Appendix A.3 (p. 18, Table 9).

Table R2. LIBERO success and recorded training and collection costs on four A800 80GB GPUs. Matched training budget denotes matched aggregate optimizer updates and global batch size. All methods use the same trajectory curation.

MethodTraining budgetTraining + collection time (h)Allocated GPU-hoursLIBERO success (%)
SFT + rollouts from one policyOriginal training budget9.638.481.4
SFT + rollouts from one policyMatched training budget24.899.282.0
VLA-CollectMatched training budget24.899.288.9

Extending single-policy training improves success from 81.4% to 82.0% (+0.6 percentage points), while VLA-Collect reaches 88.9%, retaining a 6.9-percentage-point advantage. This comparison supports the benefit of collective generation under matched optimization budgets. At deployment, VLA-Collect uses a single student with the same architecture as the underlying backbone.

(c) Persistent view assignment versus shared augmentation.

We sincerely thank the reviewer for suggesting this comparison. We compared our fixed view assignment with a pool of five experts that all sample from the same augmentation mixture. Both conditions retain WeightCoAct and CoTC and use the same initialization, initial demonstrations, augmentation types and strengths, and aggregate exposure to original observations and each augmentation family. We also matched training updates, the total budget of 100 candidate rollouts per task, the retention schedule, rollout input processing, and student training. The comparison changes how visual views are allocated across experts.

Table R3. LIBERO and LIBERO-Plus success with different augmentation assignments. Both configurations use five experts, the same WeightCoAct regularization, and the same trajectory curation. Scores evaluate the final student.

Augmentation assignmentLIBERO success (%)LIBERO-Plus success (%)
All visual augmentations for every expert87.051.5
One fixed visual configuration per expert88.954.3

The shared-augmentation control achieves 87.0% on LIBERO and 51.5% on LIBERO-Plus. Fixed assignment reaches 88.9% and 54.3%, improving success by 1.9 and 2.8 percentage points, respectively. These results support the benefit of maintaining distinct visual training distributions across experts under the same augmentation and training budgets. The protocol, results, and discussion have been added to Appendix A.6, “Fixed View Assignment versus Shared Augmentation” (p. 19, Table 12).

As a complementary control, a single policy trained with all visual augmentations and the same trajectory curation achieves 49.8% on LIBERO-Plus, compared with 54.3% for VLA-Collect, a 4.5-percentage-point advantage under matched training budgets. This comparison supports the broader contribution of collective generation with the same visual augmentations and trajectory curation. The single-policy control is documented in Appendix A.4 (p. 18, Table 10).

Q3: Success filtering versus ranking within successful rollouts.

We sincerely thank the reviewer for highlighting the distinction between success filtering and ranking. Yes, the configuration without CoTC in Table 5 already excludes failed trajectories and randomly selects successful ones under the same retention budget. Table 5 reports the full two-round pipeline, in which the first selection supplies the next expert-training round.

To isolate the benefit of ranking, we compare random and ranked selection from a fixed pool of successful trajectories collected by the same first-round experts with WeightCoAct and DataCoAct enabled. Each method retains 10 successful trajectories per task. We train a separate student on each selected subset together with the same five initial demonstrations per task, using the same student-training and evaluation settings. This control ends with student evaluation after the first collection; it includes no second expert-training or collection round.

Table R4. LIBERO success of students trained using random versus ranked selection from the same first-round successful rollout pool. Each method retains 10 successful trajectories per task and combines them with the same five initial demonstrations for student training.

Selection among first-round successful rolloutsStudent LIBERO success (%)
Random selection79.1
Ranked selection (CoTC)82.7

CoTC ranking improves LIBERO success from 79.1% to 82.7%, a gain of 3.6 percentage points. With the successful candidate pool and retained-data budget held fixed, this result supports the benefit of ranking successful trajectories for student training. We report this control in Appendix A.1 (p. 17) and clarify success filtering in the Table 5 caption (p. 8) and Appendix B.6 (pp. 20–21).

The comparisons support deliberate expert diversification and fixed view assignment under matched budgets. The first-round control further supports CoTC ranking as a way to select more effective training data on LIBERO.

We sincerely thank the reviewer again for the thoughtful comments and constructive suggestions. This feedback has helped us clarify the contributions of expert diversification and trajectory selection, strengthen the experimental controls, and improve the overall quality of our manuscript.

Reviewer_Vm3SOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies few-shot post-training of VLA models when only a handful of expert demonstrations are available but environment rollouts and binary success feedback are cheap. The authors train a pool of experts diversified through a parameter-distance regularizer on the action heads (WeightCoAct) and a fixed visual augmentation per expert (DataCoAct), filter and rank the pooled rollouts with Collective Trajectory Curation (CoTC), and distill the retained trajectories into a single student policy. They report gains on LIBERO, LIBERO-Plus, five RoboTwin tasks and five real-world tasks.

Critical strengths

The matched-budget controls in Appendix A.1 are convincing: single-policy self-training with the same interaction schedule and the same CoTC reaches 81.4, versus 88.9 for the full method, so the gain from expert heterogeneity is isolated from curation and from extra interaction.

Distilling into a single student (88.9) outperforms action-ensembling the same five experts (85.1) at one fifth of the inference cost, and the curated data transfers to six other backbones.

Critical weaknesses

The method is essentially rejection-sampling self-imitation with an augmented ensemble, and the paper does not compare against the obvious competitor: RL-style post-training using the same binary success signal (e.g., residual-RL self-improvement, which the paper itself cites). Without this, it is unclear that collective generation is the right way to spend 100 labeled rollouts per task.

The “few-shot” framing is misleading. The headline +13.7 over Qwen-OFT mixes the method with 100 extra success-labeled rollouts per task, while the ICL and PEFT baselines use none. The honest baseline is the 81.4 control, i.e., a +7.5 gain.

Compute is never matched or reported. Five experts over three rounds against one policy, and the K sweep does not control total training steps. Appendix B.9 defers this to a future code release.

WeightCoAct is under-analyzed: λ and the realized parameter distances are not reported, and parameter distance is never linked to actual behavioral diversity between experts.

The DataCoAct views (viewpoint, illumination, scene appearance, sensor noise) match the LIBERO-Plus perturbation categories, so part of the zero-shot gain is plain augmentation. The pooled-views single-policy control is reported only on LIBERO.

RoboTwin is weak evidence: tasks are selected to have 50–70% initial success, the interaction budget is not stated, and the ICL/PEFT/Retrieval rows differ from Qwen-OFT by a constant offset on every task, which needs explanation.

Questions for authors

Please add an RL-based post-training baseline using the same demonstrations and the same binary-reward rollouts.

Please report a compute-matched single-policy baseline (same total rollouts and training steps) and the pooled-views control on LIBERO-Plus.

What are λ and the action-head distances in practice, and how much do the experts actually disagree behaviorally?

Why do the RoboTwin baseline rows shift by a constant on all five tasks, and what rollout budget was used there?

3 comments

We sincerely thank the reviewer for the time and care devoted to assessing our work and for the thoughtful, constructive feedback. These comments have helped us strengthen the experimental comparisons and improve the clarity and rigor of our manuscript. We particularly appreciate the reviewer's recognition of the controls under matched interaction budgets and the consolidation of collective experience into a single student. We are equally grateful for the careful scrutiny of our claims: the distinction between demonstration efficiency, additional interaction, and computational cost is essential to a fair assessment of the method. The comment on gain attribution helped us identify and correct an ambiguity in the presentation of our main results. We address the six concerns below, grouping the corresponding questions with each weakness.

W1 & Q1: Missing comparison with RL-based post-training.

We thank the reviewer for suggesting an RL post-training comparison using the same demonstrations and binary success feedback. To address this, we added Qwen-OFT + GRPO as a direct RL post-training baseline. The experimental protocol, results, and discussion are now included in Appendix A.2, “RL Post-Training under Limited Interaction” (p. 17, Table 8), with all additions marked in blue.

Regarding the cited residual-RL approach, PLD [1] learns task-specific residual policies, collects trajectories, and distills them into the VLA. Its published configuration uses 100 warm-up episodes (Appendix C.1, Table 5), already equal to our full per-task rollout budget. We therefore chose direct GRPO for a comparison using five initial demonstrations per task and binary feedback under a comparable total interaction budget.

Each GRPO run starts from a Qwen-OFT 2B checkpoint fine-tuned on five expert demonstrations per task and uses binary task-completion rewards. We trained one checkpoint per suite (Spatial, Object, Goal, and Long), using two online training iterations, each collecting 64 groups × 8 trajectories = 512 rollouts. This gives 1,024 rollouts per suite and 4,096 overall, averaging 102.4 per task without a fixed task-level quota. CoAct uses 100 candidate rollouts per task, or 4,000 overall, and jointly trains across the four suites to produce one student checkpoint.

Table R1. LIBERO success rates (%) and additional training interactions. All methods use five initial demonstrations per task.

MethodSpatialObjectGoalLongAvg.Training rollouts, total
Qwen-OFT initial SFT87.076.070.467.275.20
Qwen-OFT + GRPO92.284.475.274.681.64,096
VLA-Collect93.895.086.680.088.94,000

GRPO improves average success from 75.2% to 81.6% (+6.4 percentage points), showing that direct RL provides meaningful gains in this setting. CoAct produces a VLA-Collect student achieving 88.9%, a further 7.3 percentage points with slightly fewer total rollouts. These results strengthen the evidence for CoAct under limited interaction.

Reference

[1] Wenli Xiao et al. Self-Improving Vision-Language-Action Models with Data Generation via Residual RL. ICLR, 2026.

W2: Few-shot setting and attribution of the reported gains.

We sincerely thank the reviewer for this insightful comment, which has helped us make the experimental comparison more precise. We particularly appreciate the reviewer’s suggestion to make this distinction explicit alongside the main results. Here, “few-shot” refers to five initial expert demonstrations per task; CoAct additionally uses 100 candidate rollouts per task with binary success feedback, whereas initial Qwen-OFT SFT, ICL, and PEFT do not.

Following this valuable suggestion, we have revised Section 4.1.1 and the Table 1 caption (p. 6), and added a clarification in Appendix A.1 (p. 16), with all changes marked in blue. The revised text distinguishes +13.7 percentage points (75.2% → 88.9%) for the full adaptation procedure, including additional interactions, from +7.5 percentage points (81.4% → 88.9%) over SFT with rollouts from one policy under the same interaction and retention schedule and the same trajectory curation (CoTC) procedure. As the reviewer rightly emphasizes, the latter is the appropriate comparison for assessing collective generation under these controls; it now appears alongside the main result. We are grateful for the reviewer's guidance in making the scope of our claims clearer.

W3 & Q2(a): Comparison under matched optimization budgets.

We sincerely thank the reviewer for this insightful suggestion, which motivated a stronger experimental control and helped clarify the contribution of collective generation. We extended SFT with CoTC-selected rollouts from one policy to match CoAct's aggregate optimizer-update budget across the expert pool and final-policy training, keeping the global batch size, 100 candidate rollouts per task, the CoTC procedure, and the data-retention schedule the same. The protocol, results, and discussion have been added to Appendix A.3 (p. 18, Table 9), with all additions marked in blue.

Table R2. LIBERO success under the original and matched training budgets. Matched training budget denotes matched aggregate optimizer updates and global batch size. All methods use the same trajectory curation.

MethodTraining budgetLIBERO success (%)
SFT + rollouts from one policyOriginal training budget81.4
SFT + rollouts from one policyMatched training budget82.0
VLA-CollectMatched training budget88.9

Extending single-policy training improves success from 81.4% to 82.0% (+0.6 percentage points), while VLA-Collect reaches 88.9%, retaining a 6.9-point advantage under the same aggregate update budget. The modest gain from longer single-policy training is consistent with limited behavioral coverage in its rollout pool, highlighting the value of complementary trajectories generated by multiple experts. The experiments with different numbers of experts in Appendix C.4 show diminishing gains as the number of experts increases, with the largest improvement from three to five experts, supporting our default choice of K=5.

W4 & Q3: WeightCoAct hyperparameters and behavioral diversity.

We thank the reviewer for requesting these details. Appendix C.6, “WeightCoAct Configuration and Parameter Distances” (p. 26; Table 22) now reports the coefficient schedule and measured parameter distances.

The effective coefficient increases linearly from 0.05 to 0.15 over the first 300 training steps and remains at 0.15 thereafter. Both the backbone and action head are trained; the regularizer applies only to the action head. Distances average per-tensor RMS parameter differences, excluding normalization and bias parameters.

Table R3(a). Mean action-head parameter distances.

MetricValue
Mean distance from experts to the fixed anchor0.020
Mean distance over distinct expert pairs0.018

We also compare WeightCoAct disabled, the default schedule, and twice the default strength (Appendix C.6, pp. 26–27; Table 23 on p. 27). All three configurations use five experts with DataCoAct enabled and CoTC disabled, keeping the remaining settings fixed. Action difference is the mean pairwise L2 distance between expert actions on the same observations.

Table R3(b). WeightCoAct strength, action differences, and final-student success.

WeightCoAct strengthAction differenceStudent LIBERO success (%)
Disabled0.1981.4
Default0.3384.2
Double0.4182.6

The default strength increases action difference from 0.19 to 0.33 and success by 2.8 percentage points. Doubling the strength further increases action difference to 0.41, while success falls to 82.6%; the default performs best among the tested settings. Table 5 (p. 8) also reports a LIBERO-Plus gain from 46.9% to 50.3% (+3.4 points) when adding WeightCoAct under the same component configuration.

Figures 5 (p. 9) and 7 (p. 29) provide qualitative examples of varied approach directions, contact regions, and execution paths in the full CoAct pool for plate pushing, drawer opening, and pot grasping. Scene, object layout, initialization, instruction, and rollout budget are fixed across methods.

W5 & Q2(b): Data augmentation and collective generation on LIBERO-Plus.

We sincerely thank the reviewer for this helpful observation, which motivated a direct comparison of collective generation with a single policy given the same visual augmentations. We trained the single-policy control on a mixture of all five view configurations used by DataCoAct, with the same CoTC procedure, five initial demonstrations per task, 100 candidate rollouts per task, and matched training budgets. The resulting policy was evaluated directly on LIBERO-Plus using the same evaluation protocol as VLA-Collect. The protocol, results, and discussion have been added to Appendix A.4, “Visual Augmentation and Collective Generation on LIBERO-Plus” (p. 18, Table 10), with all additions marked in blue.

Table R4. LIBERO-Plus success with the same visual augmentations and trajectory curation.

MethodLIBERO-Plus success (%)
SFT + rollouts from one policy (all visual augmentations)49.8
VLA-Collect54.3

The single-policy control achieves 49.8%, while VLA-Collect reaches 54.3%, a 4.5-percentage-point advantage under matched training budgets. With visual augmentation and CoTC shared, this result supports the contribution of collective generation to transfer robustness.

W6 & Q4: RoboTwin task selection, interaction budget, and result regularity.

We sincerely thank the reviewer for the careful examination of our RoboTwin experiments and the helpful suggestions on task selection, interaction budgets, and result consistency. We have added the task-selection rationale and training interaction budget to Appendix B.8, “RoboTwin Protocol” (pp. 21–22, Table 15 on p. 22), with all additions marked in blue. The initial Qwen-OFT policy is trained on 20 clean demonstrations for each of all 50 tasks. We focused on tasks with initial success rates of 50%–70% because the policy can generate sufficient successful trajectories for subsequent training while still leaving substantial room for improvement. CoAct is applied to five tasks selected from the eight within this range, improving average success from 59.6% to 78.0%.

As detailed in Table R5, post-training uses two expert-training rounds, each followed by rollout collection, and then trains the student directly. The five experts share 100 candidate rollouts per task, including successful and failed attempts, for 500 training rollouts across the five tasks. The 20 initial demonstrations remain available throughout training. The two expert-training rounds use 0 and 10 retained rollouts, respectively; student training uses 20 retained rollouts. The student therefore uses 20 demonstrations and 20 retained rollouts, with no third expert-training round or further collection. This training budget is separate from the 100 valid evaluation episodes per task and method.

Table R5. RoboTwin post-training schedule per task. Retained rollouts are the total rollout trajectories included in each stage's training set.

StageInitial expert demonstrationsRetained rollouts in training setNew candidate rollouts
Expert round 120050
Expert round 2201050
Student training20200

We conducted an additional evaluation with a different seed. Both evaluations use policy rollouts following RoboTwin's official evaluation procedure, with 100 valid episodes per task and method. ICL uses the same Qwen-OFT weights and adds demonstration context at inference; its success rates remain close to Qwen-OFT in both evaluations. Table R6 reports the repeated evaluation in the same task order, showing task-dependent differences while preserving the overall trend across the baselines.

Table R6. RoboTwin success rates (%) under an additional evaluation seed.

MethodGrab rollerMove playing cardPlace can in basketPress staplerStack three blocks
Qwen-OFT6859615553
VLA-ICL6759605453
VLA-PEFT6962635755
VLA-Retrieval7265676259

We sincerely thank the reviewer again for the careful review and constructive suggestions. The additional comparisons and clarifications prompted by these comments have strengthened the experimental evidence and improved the clarity and rigor of our manuscript.

Comments on the paper

2 comments
Area_Chair_urLbOpenReview ↗

Rebuttal Discussion

Dear Reviewers,

The authors have now submitted their rebuttal and responded to the concerns raised in the reviews. We would greatly appreciate it if you could take some time to review their responses and participate in the discussion, particularly regarding whether your concerns have been adequately addressed.

Thank you very much for your time and valuable contributions!

We sincerely thank all reviewers for their valuable time, careful evaluation of VLA-Collect, and insightful, constructive feedback. Your thoughtful suggestions have been invaluable in improving the clarity and rigor of our manuscript. We have carefully addressed each concern below and would be sincerely grateful for any further guidance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.