- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SVA (Search, Value, and Act), a test-time action-selection framework for VLA models. The authors first use MCTS in simulation to explore candidate trajectories and obtain return targets, then distill these targets into a lightweight Q-value evaluator. At deployment, the VLA policy remains fixed, generates multiple candidate actions, and the evaluator selects the candidate with the highest uncertainty-regularized value. Experiments cover embodied reasoning, simulated manipulation, test-time scaling, and five real-robot tasks.
Critical strengths
1.The paper addresses an important problem: improving VLA performance without modifying the policy backbone. Separating action generation from action evaluation is a sensible and practically relevant direction.
2.The experimental coverage is strong. The method is evaluated across several benchmarks, action spaces, proposer backbones, and model scales, together with real-robot experiments.
3.The controlled comparisons and ablations are useful. The paper examines MCTS supervision, learned value evaluation, multi-candidate selection, search-data scale, and candidate-budget scaling.
4.The test-time scaling results are interesting. In particular, the comparison between a smaller model with SVA and a larger single-shot model suggests that additional evaluation compute can be an effective alternative to parameter scaling.
Critical weaknesses
1.The pass@k analysis does not fully establish an “action evaluation bottleneck.” Higher pass@k only shows that some complete rollouts succeed under repeated sampling; it does not demonstrate that failures primarily arise from selecting the wrong action among locally available candidates.
2.The methodological novelty is somewhat limited relative to recent work on verifier-guided VLA inference, value-guided action selection, and test-time sampling. The main distinction is using MCTS-generated return targets to train the evaluator, but the conceptual gap from related approaches could be sharper.
3.The claim that SVA preserves generalist capability is not directly evaluated. Freezing the VLA parameters prevents parameter forgetting, but does not by itself show that the overall SVA system preserves performance across the policy’s broader pretrained capabilities.
4.SVA relies on substantial simulator-based search supervision and benchmark-specific evaluators. This limits the generality of the approach in settings without resettable simulators or task reward signals, and the cost of collecting MCTS supervision is not directly compared with alternative post-training approaches.
5.The real-robot results are encouraging, but both the proposer and evaluator are adapted using 500 real-robot trajectories. Therefore, these experiments demonstrate deployment-time frozen selection after adaptation rather than direct zero-shot transfer of the simulation-trained evaluator.
Questions for authors
1.How does the pass@k experiment distinguish an action-evaluation bottleneck from stochasticity or compounding execution errors across complete rollouts?
2.What is the main technical distinction between SVA and existing value- or verifier-guided VLA action-selection methods beyond the use of MCTS-generated supervision?
3.Can the authors clarify what empirical evidence supports the claim that SVA preserves generalist capability, beyond keeping the VLA parameters frozen?
4.How much simulator interaction and compute is required to generate the MCTS training data, and how does this cost compare with the post-training alternatives discussed in the paper?
5.In the real-robot study, how much of the improvement depends on adapting the Q-model with the 500 real trajectories rather than transferring the simulation-trained evaluator directly?
4 comments
Author Response (Part 2/3)
W3 & Q3: Multi-task evidence and the scope of capability-preservation claims.
A3. We assess capability preservation by comparing the full SVA system with the same frozen proposer. The proposer’s existing capabilities provide a common starting point, allowing us to assess how evaluator-guided selection changes performance across tasks. Our empirical support therefore comes from evaluating the full proposer-plus-evaluator system, not from parameter freezing alone.
Shared Q-evaluators improve average multi-task performance in the evaluated settings. A single Q-evaluator is jointly trained across all 12 RoboTwin tasks; each EmbodiedBench suite likewise uses one Q-evaluator across its task categories. On RoboTwin (Table 2), SVA improves on 11 of 12 tasks, raising average SR from 62.72% to 67.78%. On both EmbodiedBench suites (Table 1), the Q-evaluator is reused across five proposer backbones, with positive average gains for all five.
We further evaluate SVA on three RoboTwin tasks that the Q-evaluator never saw. place_shoe, place_object_scale, and shake_bottle are excluded from all search and Q-evaluator training data. We use the same frozen checkpoint, the 12-task Q-evaluator without further training, and the Table 2 protocol (three runs × 50 episodes per task, 150 trials per task and method). On these unseen tasks, SVA leaves the frozen proposer’s average success unchanged (86.7% for both). The Q-evaluator therefore does not degrade the proposer on tasks outside its training data.
| Unseen Task | + SVA | |
|---|---|---|
| Place Shoe | 87.3% | 88.7% |
| Place Object Scale | 73.3% | 71.3% |
| Shake Bottle | 99.3% | 100.0% |
| Average | 86.7% | 86.7% |
W4(a) & Q4: Offline supervision cost and comparison with post-training.
A4(a). SVA’s full offline cost on RoboTwin is about 13 hours on 4 × RTX 4090. On simulator interaction, MCTS runs one search per training episode with 128 simulations. Each simulation expands one node and evaluates it with a frozen-policy rollout. Appendix D.2 give the full search settings; the measured resource usage is below. Q-evaluator training fits the Qwen3.5-0.8B backbone with LoRA and five value heads on all of these samples. Both stages are fully automated, and neither updates the VLA backbone. GPU memory is the peak usage per GPU.
| Stage | Hardware | Wall-clock time | GPU memory |
|---|---|---|---|
| MCTS data collection | 4 × RTX 4090 | ≈ 12 h | ≈ 22 GB |
| Q-evaluator training | 4 × RTX 4090 | ≈ 1 h | ≈ 18 GB |
Published RL post-training methods report larger training setups, although the settings are not directly comparable. The table below lists the training configurations reported by recent RL post-training works. None of them reports total GPU-hours, and they differ from our setting in hardware, backbone, task set, and initial demonstrations. The table is therefore not a controlled comparison, and we do not derive a cost ratio from it.
| Method | Backbone / trained parameters | Reported hardware | Reported training data and interaction |
|---|---|---|---|
| SimpleVLA-RL [5] | OpenVLA-OFT, full-parameter training | 8 × A800 80GB | RoboTwin 2.0, 12 tasks: SFT on 1,000 demonstrations per task, then RL on 1,000 simulation scenarios per task with 8 sampled rollouts each |
| RLinf-VLA [6] | OpenVLA-OFT, LoRA (GRPO) | 8–32 × H100 80GB | RoboTwin, 6 tasks with one task-specific model each: 1,000 fixed scene seeds per task, 128 parallel environments, group size 8 |
| [7] | / , 300M action expert (VLM frozen) | 8 × H100 80GB | LIBERO: 500 training epochs, 64 parallel environments, 8 rollout epochs per training epoch |
| SVA (ours) | Q-evaluator only (LoRA + heads) | 4 × 4090 24GB | RoboTwin 2.0, 12 tasks: 30 search episodes per task, no additional demonstrations; ≈ 13 h in total |
These RL methods update the policy using about 1,000 training scenarios per task on data-center GPUs, whereas SVA’s offline stage uses 30 search episodes per task on four consumer GPUs and leaves the proposer unchanged. The two approaches are complementary rather than interchangeable: RL changes what the policy generates, SVA changes how its candidates are selected, and an RL-fine-tuned policy can serve as an SVA proposer.
[5] SimpleVLA-RL: Scaling VLA training via reinforcement learning. ICLR 2026.
[6] RLinf-VLA: A unified and efficient framework for reinforcement learning of vision-language-action models. arXiv:2510.06710, 2025.
[7] : Online RL fine-tuning for flow-based vision-language-action models. arXiv:2510.25889, 2025.
Author Response (Part 3/3)
W4(b): Applicability without resettable simulators.
A4(b). SVA's search stage requires a resettable simulator and a task-success signal; deployment does not. On RoboTwin the signal is a sparse success indicator from the built-in checker, without reward shaping. Deployment uses only candidate generation and Q-evaluator scoring, without simulator access.
When a simulator is available, search supervision is inexpensive. Collection is fully automated, with no human teleoperation or physical resets: RoboTwin MCTS data (30 episodes per task) took about 1 hour per task on 4 RTX 4090 GPUs, whereas teleoperating 100 real demonstrations took about 1.5 hours of operator time per task. For the physical experiments we reused existing RoboTwin 2.0 tasks and aligned only the camera intrinsics/extrinsics and the spacing between the two arms with our platform, once per robot. A task outside RoboTwin would require building a simulated task with a success checker.
Without a simulator, the Q-evaluator can be trained or adapted on independently collected rollouts that contain both successes and failures. The regression objective only needs observed returns for executed state–action pairs, so real-robot rollouts labelled with task success or failure can supervise the Q-evaluator without tree search or intermediate-state restoration. On RoboTwin, Rollout-Q trained on such independent rollouts already improves success over the base policy (64.17% versus 62.72%), and MCTS raises it further to 67.78% when a simulator is available.
W5 & Q5: Quantifying Q-evaluator adaptation versus direct transfer.
A5. Table 4 evaluates frozen-policy selection after adaptation, not zero-shot transfer. With the same real-adapted proposer, the adapted Q-evaluator raises SR from 42.8% to 55.6% (+12.8 points); this alone does not isolate the contribution of real-data adaptation.
Most of the gain comes from direct transfer of the simulation-trained Q-evaluator. All conditions use the same frozen, real-adapted proposer, the same candidate settings, and the same protocol (five tasks, 25 trials per task under each of the target-only and with-distractors conditions). MCTS-only denotes an evaluator with no real-data fine-tuning; the proposer is still real-adapted. Without adaptation it raises SR from 42.8% to 50.4% (+7.6 points); adaptation on the 500 real demonstrations adds 5.2 points, reaching 55.6%. With 250 trials per condition, these are point estimates with substantial sampling uncertainty.
| Condition | Q-evaluator training | SR (%) |
|---|---|---|
| Base policy | No reranking; execute the common real-adapted proposer | 42.8 |
| MCTS-only | Trained on simulation MCTS data, without real-data adaptation | 50.4 |
| MCTS pretraining + real adaptation | The same Q-evaluator adapted on the 500 real demonstrations | 55.6 |
Adaptation reuses existing demonstrations without additional real-robot collection. We reuse the 500 expert demonstrations (100 per task) already collected for the common proposer, assign reward 1 at the final step of each trajectory with returns discounted backward by 0.99, and fine-tune the LoRA adapters and value heads of the simulation-trained Q-evaluator by Smooth-L1 regression (learning rate , batch size 64, 5 epochs).
Summary Response
We thank you for your constructive review. (A1) At 35% of RoboTwin initial states some of the eight candidates succeed and others fail; a random choice succeeds 73.33% of the time versus 88.33% for at least one candidate, an optimistic gap that motivates the evaluator but does not by itself establish the bottleneck. (A2) SVA outperforms TACO and V-GPS with the same proposer and candidate budget, and MCTS adds 3.61 points over rollout returns with the same number of training samples. (A3) On three tasks unseen by the Q-evaluator, average success is unchanged (86.7%). (A4) The offline cost is about 13 hours on four consumer GPUs; the simulator-free rollout route gives a smaller gain and is untested on a real robot. (A5) On the real robot, the simulation-trained Q-evaluator alone raises SR from 42.8% to 50.4%. We hope these clarifications and new evidence fully address your concerns, and we would sincerely appreciate your reconsideration of our work.
The authors’ rebuttal has addressed most of my questions and clarified several aspects of the evaluation and methodology. While some limitations remain, the response improves my understanding of the paper. I therefore keep my original rating unchanged.
Author Response (Part 1/3)
W1 & Q1: Interpreting pass@k and establishing the action-evaluation bottleneck.A1. Pass@k shows that the frozen policy can succeed, not why it fails. We agree that a higher pass@k cannot distinguish a wrong action choice from rollout stochasticity or accumulated execution errors, and we do not claim that most failures stem from action selection. Our claim is narrower: choosing among the proposer's own candidates at a given state is a substantial and addressable source of failure.
State-level analysis. In the RoboTwin π0.5 search data, a candidate is successful if executing it and then rolling out the frozen policy reaches task success. Of the 360 initial states, 35% contain both successful and failed candidates. Choosing one of eight candidates at random succeeds 73.33% of the time, while at least one candidate succeeds in 88.33% of states (a 15-point gap); across all 3,357 states with eight evaluated candidates the figures are 79.91% and 88.62%. Each label comes from a single stochastic rollout, so this gap is an optimistic estimate of what selection can achieve. It motivates an evaluator but does not by itself show that selection is the bottleneck; the evidence is the downstream gain of the learned Q-evaluator over other selectors on the same frozen proposer (A2).
Same-state case studies. In EmbodiedBench (Table 6, Fig. 10), the goal-directed plan has a lower prior than the distractor-driven plan (0.0625 vs. 0.4375) but a higher Q-value (0.2948 vs. −0.1282); SVA selects it and succeeds. On the real robot (Fig. 7), at step 48 of
pick_dual_bottlesSVA scores 16 action chunks from the same state and selects the one with the highest Q-value (0.80); the episode succeeds at step 95, while the baseline reaches the 200-step limit. These cases are illustrative, not statistical evidence.W2 & Q2: Technical distinctions beyond MCTS-generated supervision, and novelty beyond recent work on verifier-guided VLA inference.A2. SVA bridges training-time search and test-time scaling through learned action evaluation. It distills the outcomes of simulation-based search into a lightweight Q-evaluator, enabling frozen embodied policies to select actions based on their downstream consequences without online simulator search. This places SVA between two lines of prior work. Learned selectors score candidates cheaply at test time but are trained without search: TACO [1] ranks candidates by learned pseudo-counts, V-GPS [2] learns a Q-function through offline RL, and RoboMonkey [3] trains an action preference model on synthetic preference data. Search-at-deployment methods, such as value-guided tree search with V-VLAPS [4] or rollouts in a learned world model, look ahead at test time but pay the search cost at every decision. SVA moves search to training time. Because the search branches over the frozen proposer's own candidates, sibling candidates from the same state receive Monte Carlo return labels that include both successful and failed outcomes. The Q-evaluator regresses these returns without temporal-difference bootstrapping, so it learns the same comparison it performs at deployment, where compute scales only with the number of scored candidates.
On RoboTwin, SVA outperforms both learned selectors with the same frozen proposer and candidate budget (mean ± SD of the 12-task average over three evaluation runs of 50 episodes per task, fixed checkpoints).
A controlled ablation isolates the effect of search supervision. Rollout-Q and MCTS-Q use the same 30 training episodes per task, the same number of training samples (92,728), the same Q-evaluator architecture and hyperparameters, and the same frozen proposer, and are evaluated on identical test episodes and candidate settings. Only the supervision differs: returns from independent sequential rollouts versus MCTS-backed returns. Rollout-Q reaches 64.17%, close to TACO (65.33%) and V-GPS (64.72%), whereas MCTS-Q reaches 67.78% (+3.61 points) and is higher on 11 of 12 tasks. Search adds same-state success/failure contrast among sibling candidates and concentrates the budget on promising branches; the ablation measures their combined effect.
[1] Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach. arXiv:2512.02834, 2025.
[2] Steering your generalists: Improving robotic foundation models via value guidance. CoRL 2024.
[3] RoboMonkey: Scaling test-time sampling and verification for vision-language-action models. CoRL 2025.
[4] V-VLAPS: Value-guided planning for vision-language-action models. arXiv:2601.00969, 2026.