acceptodds
Under review as a conference paper at ICLR 2027

Verify, Act, Distill: Turning Test-Time Compute into Test-Time Adaptation for Vision-Language-Action Models

Abstract

Test-time compute (TTC), which scores multiple candidate actions at inference time, is a powerful way to improve the execution performance of vision-language-action (VLA) models, but existing methods keep the policy frozen, so the gain must be recomputed at every execution. We propose Verify, Act, Distill, a framework that consolidates the gains of TTC into the VLA itself. A fixed verifier scores candidates generated by a flow-matching VLA, the weighted aggregate is executed, and the observations and aggregated chunks from successful adaptation episodes are distilled directly into LoRA parameters with the standard flow-matching loss. This reuses supervision that TTC has already produced during execution, without additional reinforcement learning or rollouts dedicated to teacher generation. On the 40 LIBERO tasks with a SmolVLA 5k checkpoint, distilling from only 10 episodes per task raises the mean success rate of verifier-free single-candidate generation from 54.1% to 70.6%, retaining about 74% of the improvement obtained by aggregating eight candidates. Applying the same aggregation to the distilled policy further raises the success rate from 76.4% to 82.3%, with a particularly large gain on LIBERO-Long (44.0% to 57.8%), and the single-candidate policy also improves on LIBERO-plus, which contains distribution shifts. These results show that using verification as a learning signal, rather than only as an execution-time selector, both retains the improvements of TTC and re-amplifies the effect of subsequent TTC.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Test-Time Adaptation, Test-Time Computation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yMyxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies whether gains from test-time computation can be retained in VLAs without repeatedly using a verifier. The authors propose a test-time adaptation framework that fine-tunes the VLA using aggregated actions from successful episodes, with action candidates scored by a pre-trained verifier. On the LIBERO benchmark, the distilled policy improves single-candidate success over the base policy, and reapplying candidate aggregation yields further gains with the 5k checkpoint. The additional gain from aggregation is smaller with the 10k checkpoint and under distribution shift, as evaluated on LIBERO-Plus.

Critical strengths

None

Critical weaknesses
  • My main concern is that the paper has limited methodological novelty. Test-time adaptation is a promising approach for VLAs; however, simply fine-tuning a policy on verifier-guided actions from successful episodes seems too straightforward and is already a common approach in other domains. The verifier training also largely follows RoboMonkey. Moreover, the authors do not provide a clear reason for choosing weighted-average execution over selecting the highest-scoring candidate, i.e., simple Best-of-N selection, in the proposed Act stage.
  • I think the paper’s empirical evaluation and presentation are incomplete. First, the method’s effectiveness beyond the evaluated SmolVLA and LIBERO setups remains unclear. On LIBERO-plus, the additional gain from applying aggregation to the distilled policy over Agg-8 is small, making the evidence for this combined effect narrow. Comparisons with prior test-time adaptation works are also limited: N-GRPO in main Table 1 is the authors’ own implementation, which I do not find convincing as the main baseline. Second, the presentation makes the paper really difficult to follow. Implementation details are mixed into the method section, and some important ablation studies (Table 5) are placed in the appendix rather than the main text.
Questions for authors

Q1. Why do the authors use N-GRPO as the main baseline in Table 1? It applies GRPO in noise space, whereas applying GRPO to update the VLA policy itself seems like a more direct comparison with the proposed method. Can the authors provide results for such a baseline under the same experimental setup as Table 1?

Q2. For the Table 1 experiment, Distill + Agg-8 improves 24 tasks but performs worse on 10 tasks relative to Agg-8. Can the authors provide any explanation or hypothesis for these results? I think the results are directly relevant to the method’s effectiveness.

Q3. Can the authors provide any experimental results showing that the proposed method works with base policies other than SmolVLA or on simulation benchmarks beyond LIBERO?

Q4. Can the authors provide a sensitivity analysis of the proposed test-time adaptation method to verifier quality? The method seems to rely heavily on the verifier because it guides action aggregation, which affects rollout success and, in turn, which episodes enter the success-filtered fine-tuning buffer.

Reviewer_piH6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Verify, Act, Distill, which adapts a VLA by collecting successful executions under verifier-guided candidate aggregation and fine-tuning task-specific LoRA parameters with a standard flow-matching loss. The adapted policy can subsequently operate without the verifier or reuse candidate aggregation. Experiments on SmolVLA and LIBERO shows distillation retains TTC gains and improves subsequent TTC performance.

Critical strengths
  1. Effective method: The paper addresses a practical objective: retaining expensive inference-time improvements in a cheaper policy. With ten adaptation episodes per task, verifier-free success improves from 54.1% to 70.6%, retaining approximately 74% of the original eight-candidate improvement.

  2. Extensive experiments results: The experiments include relevant controls for self-imitation, success filtering, and teacher construction. The paper also reports weaker results at stronger checkpoints and under distribution shift.

Critical weaknesses
  1. The incremental contribution over established improved-action distillation remains unclear. Methods such as PA-RL already connect action improvement to supervised policy updates. Here, the main distinction is reusing execution data under a fixed verifier. The self-imitation comparison does not fully ablate the benefit of aggregated teacher actions.

  2. The strongest claims have limited empirical scope. The headline gains use an incompletely trained 5k checkpoint. At 10k, single-candidate improvement decreases to 83.2% to 86.2%, while subsequent aggregation slightly underperforms the original TTC policy, 89.8% versus 90.4%. Under LIBERO-plus shifts, 66/140 tasks obtain no successful adaptation episode and therefore cannot update. Furthermore, evaluation is limited to one VLA architecture and a single training seed.

Questions for authors
  1. Does Distill retain its advantage over self-imitation when the number of successful trajectories or training boundaries is matched?

  2. Are the improvements consistent across independent adaptation seeds, particularly for stronger checkpoints?

Reviewer_qvDtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Verify, Act, Distill, a novel framework designed to integrate the performance gains of test-time compute (TTC) directly into Vision-Language-Action (VLA) models. Specifically, the method employs a fixed verifier to evaluate and score action candidates generated by a flow-matching VLA, after which a weighted aggregate of these candidates is executed. For successful adaptation episodes, the corresponding observations and aggregated action chunks are directly distilled into the VLA's LoRA parameters using a standard flow-matching loss. The primary contribution lies in effectively reusing the supervision naturally produced by TTC during execution, completely eliminating the need for additional reinforcement learning algorithms or dedicated rollouts for teacher data generation.

Critical strengths

1.A simple and appealing bridge from TTC to policy adaptation. Reusing the aggregated action chunks already created for execution avoids adding a separate RL objective or teacher-generation rollout pipeline. The success-filtered distillation design is intuitive and the manuscript clearly explains when it skips an update.

2.The paper tests both retention and interaction with further TTC. It does not stop at a single-candidate result: the 5k, 10k, 20k, and distribution-shift analyses reveal when aggregation and distillation reinforce each other and when they do not. Reporting the 20k degradation cases is a meaningful strength.

Critical weaknesses

1.For VLA research, real-robot experiments are a must. Relying solely on virtual simulators like LIBERO and LIBERO-PLUS without real-world testing is a major gap. Experiments can see works like [1], which adopts libero 4 variants.

2.The evaluation is limited to a single architecture (SmolVLA), which is insufficient to prove the method's generalizability across other architectures like DiT or π\pi series.

[1] CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion

Questions for authors

See weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.