acceptodds
Withdrawn from ICLR 2027

MOSAIC: Multi-Policy Sampling and Collaborative Test-Time Adaptation for Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pretrained vision-language-action (VLA) policies integrate visual observations and natural-language instructions to generate executable robot actions, provid- ing a strong foundation for language-conditioned visuomotor control. However, large-scale pretraining and task-specific adaptation do not guarantee consistent deployment success. A single policy may produce different execution outcomes across stochastic action samples, while different policies may exhibit complemen- tary successes and failures across initial states within the same task set. These variations create opportunities to discover successful behaviors during deploy- ment. Yet a successful execution does not automatically become transferable to other policies or reliably reproducible by the policy that generated it. We there- fore investigate how collaborative adaptation can turn successful deployment-time experience into transferable and reproducible policy behaviors. We propose MOSAIC, a Multi-pOlicy Sampling And Inter-model Collaboration framework for test-time adaptation of vision-language-action models that learns from successful trajectories generated by multiple policies and sampling seeds us- ing environment-provided task-completion feedback. Starting from the same ini- tial simulator state, participating policies perform rollouts with different sampling seeds and exploit successful experience through two complementary mechanisms. Cross-policy action distillation aligns other policies’ action predictions with suc- cessful actions at the corresponding observations, facilitating behavioral transfer across policies. Within-Policy Sampling Consolidation incorporates a policy’s own successful actions into its native generative training objective, encouraging successful behaviors to recur across stochastic samples. Both mechanisms up- date only a designated module near each policy’s action output, requiring neither additional expert demonstrations nor additional trainable modules. We adapt the policies on a subset of initial states for each task, then freeze the resulting parameters and evaluate on disjoint held-out initial states. Experimental results show that across four manipulation benchmarks and 52 policy pairs, CAD improves 103 of 104 policy-side evaluations with an average gain of 3.03 per- centage points, while WTC independently improves average task success by 2.91 points.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
test-time adaptation, vision-language-action models, multi-policy sampling, mutual learning, robotic manipulat

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Cv6VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MOSAIC, which is a test-time adaptation framework for pretrained VLA policies. The paper introduces two mechnisms for the purpose. The first is Cross-Policy Action Distillation (CAD), which leverages successful trajectories to update a policy that failed on the same initial sate. The second is Within-Policy Trajectory Consolidation (WTC), which trains a policy on its own trajectories using re-noised action samples. Adaptation and evaluation use held-out initial states. Across four simulation benchmarks, CAD reports an average success-rate improvement of 3.03 percentage points, while WTC reports an average improvement of 2.91 percentage points across 22 policy–benchmark combinations.

Critical strengths
  1. CAD reports gains in 103 of 104 directions across 52 policy pairs and four simulation benchmarks. Although uncertainty estimates are missing, the breadth of these results suggests that fine-tuning on successful trajectories is useful across different VLA architectures.

  2. The adapted policies are frozen and evaluated on initial states not used for adaptation. This strengthens the evidence that the gains carry over to new initial conditions rather than just improving repeated attempts on the same instances.

Critical weaknesses
  1. The paper does not show that peer data adds anything over self-generated data. The main comparisons are against frozen policies, so they do not separate the benefit of peer data from that of collecting successful trajectories and fine-tuning on them. A comparison of peer-generated and self-generated successful trajectories under the same optimization budgets is necessary. Collaboration is the paper’s central contribution, but its advantage over self-training is currently unsupported.

  2. WTC’s claimed effect is not in the method, and the method is undefined for several evaluated models. Equation (8) averages the native loss over K noise samples. Its expectation is unchanged, and there is no cross-sample consistency term. Whether it improves repeatability is an empirical question, but Table 8 only reports average success rates, so the claim is untested. I think it's necessary to compare with ordinary fine-tuning at the same compute, measuring repeated success on fixed initial states. Separately, the diffusion formulation and re-noising procedure do not apply to standard OpenVLA-OFT’s L1 regression head or G0.5’s autoregressive action tokens, yet both appear in the results. Until the losses actually used are specified, the paper does not describe the method that produced its tables.

Questions for authors
  1. Does CAD outperform fine-tuning on the receiving policy’s own successful trajectories under matched rollout and optimization budgets?
  2. On fixed initial states with repeated seeds, does WTC improve repeated success over ordinary fine-tuning at equal compute?
  3. Did WTC use only successful trajectories or all rollouts? How were its loss and re-noising implemented for OpenVLA-OFT and G0.5?
Reviewer_j9LDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MOSAIC adapts VLA policies using trajectories collected at test time. Cross-policy action distillation transfers a successful policy’s trajectory to a policy that failed from the same initial state. Within-policy trajectory consolidation reuses a policy’s own actions under different noise samples. The evaluation covers 52 policy pairs across four simulation benchmarks, with adaptation and evaluation initial states held apart.

Critical strengths
  1. Cross-policy adaptation has a positive reported gain in 103 of 104 directions across four benchmarks, with a reported average improvement of 3.03 percentage points (Table 3). This supports the feasibility of transferring successful trajectories between heterogeneous policies without aligning their internal representations.

  2. The experiments evaluate frozen adapted policies on held-out initial states. The separate within-policy baseline and adaptation-location comparison also address alternatives to updating an entire model from a peer’s trajectories.

Critical weaknesses
  1. The experiments do not isolate the benefit of collecting another policy’s successes instead of collecting more successful trajectories from the receiving policy. WTC is a relevant comparison, but the paper does not report a comparison matched in rollout and update budgets. CAD averages over 104 policy directions, whereas WTC averages over 22 policy–benchmark entries, so their overall means are not directly comparable. Section 4.2 acknowledges the limitation and does not claim universal superiority. The remaining question is whether peer-policy collection offers a benefit at a given budget.

  2. Section 4.1 and the appendix leave the adaptation-set size, sampling attempts, update count, learning rate, and value of K for WTC unspecified. Repeated adaptation runs and paired uncertainty estimates are also not reported; Section 6 acknowledges the uncertainty limitation. Without the protocol details and variability, the roughly three-point gains and the cost of adaptation are difficult to assess.

Questions for authors
  1. On a representative subset, how does CAD compare with training on the receiving policy’s own successful trajectories when rollout and update budgets are matched? Please count unsuccessful sampling attempts and peer-policy inference in that budget.

  2. Please report the adaptation/evaluation sample counts, K, optimizer and update schedule, and checkpoint-selection rule. On the same representative subset, what are the paired uncertainty across held-out initial states, variation across adaptation runs, and measured adaptation time?

Reviewer_GaEMOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MOSAIC uses successful deployment trajectories to adapt pretrained VLA policies. Cross-Policy Action Distillation transfers trajectories from successful policies to unsuccessful peers. Within-Policy Trajectory Consolidation re-noises a policy’s own actions and trains them with the policy’s native generative objective. Adaptation primarily updates a terminal action module. Evaluation uses held-out initial states and reports an average improvement from 57.56% to 60.60% across 52 policy pairs.

Critical strengths

Adaptation and evaluation use disjoint initial states, and adapted parameters are frozen before evaluation. This tests whether improvements extend beyond the trajectories used for optimization. Evaluation across heterogeneous policies and four benchmarks suggests broad compatibility. The complete pair-level results support the reported aggregate numerical trend. Updating only the action-output module is straightforward to implement. The paper appropriately acknowledges that the existing experiments do not establish superiority over self-generated supervision.

Critical weaknesses
  1. The evaluation does not distinguish the proposed collaborative mechanisms from ordinary fine-tuning on successful trajectories. CAD uses successful observation–action data from another policy as supervised training data. WTC’s re-noising procedure closely resembles standard diffusion or flow-matching training. The experiments therefore primarily establish that additional successful data can improve a policy. They do not sufficiently establish the independent value of collaboration, success–failure pairing, or consolidation. Comparisons should control interaction counts, successful-trajectory counts, and optimization budgets.
  2. The definition of WTC is inconsistent. The abstract and experimental discussion describe consolidation of environment-validated successful trajectories. However, Section 3.2 explicitly states that WTC can be applied independently of task-completion outcome, and Equation 8 contains no success filter. These alternatives define materially different algorithms: one performs success-conditioned self-imitation, while the other may also reinforce failed behavior.
  3. Essential experimental details are missing. The manuscript does not sufficiently specify adaptation and evaluation state counts, rollouts per state, the number of re-noised samples KK, learning rates, update counts, or stopping criteria. Cross-policy handling of action normalization, chunk lengths, and history inputs is also unclear. For memory-based policies, a single observation may not be sufficient to reconstruct the native training input.
  4. The unified framework and reliability claims are incompletely evaluated. Table 3 primarily evaluates CAD, while Table 4 evaluates WTC separately. Results for their combination are not clearly presented. Moreover, the 104 directional evaluations share policies and tasks, so they should not be interpreted as independent confirmations. Several improvements are only 0.1–0.3 percentage points. Without evaluation counts and paired uncertainty estimates, the “103 out of 104” headline has limited evidential strength.
Questions for authors

Does WTC use only successful trajectories or also failed trajectories? Please provide the algorithm corresponding to the reported results. Under matched interaction and optimization budgets, how does CAD compare with self-generated successful-trajectory fine-tuning and simple pooling of successful peer trajectories? What are the results of combining CAD and WTC? Are the two mechanisms complementary? Can the authors provide the complete adaptation protocol and cross-policy action/history conversion rules?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.