acceptodds
Under review as a conference paper at ICLR 2027

From Evaluation to Recovery: Value-Guided Vision-Language-Action Models

Abstract

Vision-language-action (VLA) models learn to generate actions from demonstrations, but the imitation objective does not teach them to compare those actions. Executing a poor sample can therefore leave the robot in an unfamiliar state from which success is difficult. We introduce VaSeR, which equips an existing VLA with an action-conditioned value for selection and recovery. A value token in the policy’s own backbone scores each candidate through the vocabulary head in one forward pass. It learns from demonstrations alone, without failure data: each demonstrated action is paired with a constructed negative whose pseudo-label decreases with its deviation from the demonstration. At inference, VaSeR executes the highest-valued of several sampled candidates and records its value before execution. When these values drop or stagnate, the robot physically retreats through its recent poses and resumes. On ten RLBench tasks, VaSeR raises success from 60.2% to 76.5% on π₀.₅ and from 52.3% to 65.3% on HybridVLA; on two real-robot tasks, the corresponding improvements are from 58.3% to 73.3% and from 53.3% to 78.3%. Held-out evaluations show that constructed negatives improve both candidate ranking and episode-level alarm accuracy.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Action Evaluation, Value-Guided Action Selection, Failure Recovery, Robot Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TJ3hOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed VaSeR, which is a method that equips a VLA with an action-conditioned value token. The value token is integrated into the policy's backbone, allowing for a single forward pass computation. The role of the value token is to score demonstrations produced by the VLA. Importantly, the value is trained from demonstration data alone. The demonstration actions receive a discounted time-to-go target, while the negatives receive a lower 'pseudo-label' based on their distance to the demonstration. At inference, the value selects the best sample, and is able to trigger a retreat when its values drop / stagnate. VaSeR is evaluated on ten RLBench tasks, showing improvement over pi_0.5 and a diffusion-only HybridVLA. The authors also demonstrate the performance improvements on two real-robot tasks.

Critical strengths
  1. The idea is quite simple and cheap, requiring no additional models or data. This is quite a neat idea, as the lack of failure data is a major bottleneck for robot training. The approach is also shown to be feasible in both diffusion-based and flow-matching VLAs.

  2. The writing is clear and easy-to-follow.

  3. The paper contains useful ablations regarding the value objective (e.g., negatives / gradient detachment).

Critical weaknesses
  1. I notice that the gains seem to be concentrated around the tasks where the baseline is unusually weak. In table 1, pi_0.5 succeeds on Close Box in only 4/60 episodes, while HybridVLA reaches 70% on the same task. II believe this single task accounts for a large portion of the percentage points gained on pi_0.5. The main pi_0.5 checkpoint also seems much weaker than the earlier four-task models in Appendix B (. Could you comment on why the main baseline underperforms, and on whether VaSeR still helps a well-tuned actor?

  2. I understand you have removed the HybridVLA baseline's autoregressive branch and gated fusion, but it is arguably what defines HybridVLA. Would it be possible to evaluate the original model under your protocol?

  3. I believe it would be quite informative and important to add a few baselines. For instance, for selection, these could be RoboMonkey, V-GPS or IBC-style energy. For failure detection, they could be SAFE, Sentinel or FAIL-Detect, which you already cite. Could the authors please address why none of the relevant approaches have been baselined against your method?

  4. The paper does not seem to clarify where the 'recovery' gain comes from. As far as I can see, it could be the timing of the alarm, the physical retreat, or simply from drawing fresh samples? I could suggest (i) retreat triggered at random times with a matched alarm rate, and (ii) withholding the action on alarm and resampling without moving.

  5. I would also appreciate some intuition for why a value trained only at demonstration observations should drop in unfamiliar states, given that the negatives perturb actions rather than states. I think this would be important to clarify.

Questions for authors

Please see "Weaknesses".

Reviewer_WUHhOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

VaSeR adds an action-conditioned value token to a generative VLA, scoring a clean candidate through the shared backbone and vocabulary head. Demonstration actions receive discounted progress targets, while perturbed or policy-generated actions receive lower pseudo-labels based on their distance from the demonstrated action. Classification and ranking losses train the evaluator without failure-labeled training data. At inference, the robot selects the highest-scoring sampled candidate and monitors the selected values for drawdown or stagnation. An alarm can withhold the next action and trigger a physical retreat through one to three recorded poses, without restoring the environment state.

Critical strengths

The component controls provide meaningful evidence that the inference mechanisms, rather than an improved actor alone, produce the gains. Tables 3 and 13 show essentially unchanged single-candidate performance after value training, followed by gains from selection and from recovery with a single candidate. On the four-task subset, pi0.5 changes from 135/240 with value training alone to 173/240 with selection, 162/240 with recovery alone, and 195/240 with both; HybridVLA has the same aggregate ordering. Table 2 evaluates all three evaluators on the same VaSeR-generated candidate pairs and alarm episodes, avoiding a simple actor-sampling confound. This supports the practical usefulness of the learned evaluator more directly than the cumulative headline comparison alone.

The evidence spans two generative backbones and reports informative failures of scaling and training assumptions. Table 1 covers 600 episodes per configuration across ten tasks, and Table 4 adds two physical manipulation settings. Table 9 shows that increasing candidates to eight reduces success while increasing latency; Tables 7-8 show that shared value supervision can damage the actor and that its isolated training gain changes sign across seeds. The authors also distinguish older, non-significant fixed-depth contrasts from the main adaptive-depth results (Tables 11-12). These disclosures help identify when the method is useful without overstating universal improvements from value training, more sampling, or retreat.

Critical weaknesses

The complete method's principal results cannot be reconstructed from the main-experiment record. Appendix F explicitly states that the three seed groups do not identify training versus evaluation randomness and that the ten-task training recipe is incomplete. Table 6 is expressly restricted to older four-task models. The main checkpoints' exact negative construction, training schedule, and sampler settings are therefore not fully established; this matters because Table 7 demonstrates large performance sensitivity to negative construction. For recovery, Appendix D gives the main threshold formula but restricts the concrete command budgets, eligibility rules, planning attempts, and tolerances to an older study, explicitly declining to substitute them for missing main metadata. The real-robot section likewise lacks a complete per-run protocol and does not explain how the seven-dimensional deviation penalty and simulation keyframe target clock become the ten-dimensional, 10 Hz setting. These are parameters that define the intervention and training procedure, so the issue affects reproducibility and interpretation of the headline gains, rather than merely presentation. The paper's transparency about the missing records is appreciated, but does not resolve this limitation.

The episode-level alarm metric does not isolate the value of intervention timing in the deployed controller. Table 2 counts a failed episode as correctly detected if an alarm occurs at any step, on single-candidate, no-retreat trajectories. It supplies neither the success/failure class breakdown nor when an alarm occurs relative to loss of recoverability. The deployed controller instead monitors the maximum-scoring candidate among two, can retreat, clears history, and resumes; these operations change both the score and state distributions. The aggregate recovery gains in Tables 1 and 13 are useful evidence, but Table 11 varies depth while retaining the same alarm timing, and Appendix D explicitly says it does not test timing. Consequently, high episode-level alarm accuracy does not by itself establish the quality of intervention timing in the complete controller. The controlled recovery gains do support the usefulness of alarm-triggered retreat as a whole. The remaining uncertainty is consequential because a late alarm or unnecessary retreat consumes the same limited physical command budget as productive actions.

Questions for authors

Please supply the actual main ten-task and real-robot configurations: negative type, training updates and optimizer settings, generation sampler, command budget, retreat eligibility/repeat limit, and failed-planning handling. How were the deviation penalty and discounted progress clock adapted to ten-dimensional relative actions at 10 Hz? Can the referenced supplementary files or retained records resolve these entries?

What are the alarm confusion counts, first-alarm locations, and recovery outcomes conditional on an alarm under the actual two-candidate controller? Were calibration trajectories collected under the same selection policy, and how was the stage reference handled after history reset? Evidence distinguishing useful intervention timing from simply permitting occasional retreat could materially strengthen the recovery interpretation; this is not a request for additional unrelated tasks.

Do the three groups in Table 1 use the same trained checkpoint or separately trained models, and which sources of randomness differ? Is there an independently trained replication of the complete selection-plus-recovery method? The three-seed study in Table 8 concerns an older actor-only comparison, so it does not answer this question about the headline system.

Reviewer_N1izOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes VaSeR, a framework that integrates action value evaluation and failure recovery into a single Vision-Language-Action (VLA) model. By introducing a dedicated value token ([V]) into the VLA backbone, the model directly predicts the value of candidate actions in a single forward pass. During training, VaSeR relies solely on successful demonstrations and constructs negative samples by applying noise perturbations to expert actions, assigning lower pseudo-labels based on their action-space deviation from the demonstration. At inference time, VaSeR generates multiple candidate actions paired with their predicted values, executing the candidate with the highest value. Furthermore, if the temporal trend of predicted values becomes abnormal, the robot physically retreats through recently recorded poses for recovery. Experiments across multiple simulated and real-world tasks demonstrate performance improvements.

Critical strengths

1.Integrating action evaluation and failure recovery into a unified VLA architecture without requiring a separate external verifier model seems has practical appeal for deployment.

2.Constructing negative samples by perturbing expert actions eliminates the need to collect additional failure trajectories or rollout rewards, reducing the cost of data collection.

3.The experimental evaluation spans two distinct VLA backbones (π0.5\pi_{0.5} and HybridVLA), 10 RLBench tasks, and 2 real-robot tasks, and separately ablates the contributions of value training, candidate action selection, and backtracking recovery.

Critical weaknesses

1.The constructed negative labels may not reflect actual action quality. Robotic manipulation tasks are typically multimodal, where multiple valid actions may exist for the same state. The authors use the distance between an action and the expert demonstration to construct value labels; however, a smaller action distance does not equate to a higher success rate.

2.Some experimental results appear unfavorable to the paper's claims. For example, in Table 3, incorporating the value token even slightly degrades the policy's performance.

3.Several important experiments are deferred to the appendix, such as the test-time scaling performance as the number of candidates increases (Table 9) and the qualitative results illustrating Recovery (Figure 5).

Questions for authors

1.Although I mentioned in Strength 1 that a unified architecture looks cleaner, from the ablation study in Table 3, I do not quite see the necessity of the value token. Incorporating value training does not improve the task success rate (and even slightly degrades it), while still requiring architectural modifications to the original network and retraining. Why not adopt the common practice of using an external verifier model? That would neither compromise the test-time scaling and recovery capabilities of the overall framework nor require modifying the original VLA architecture.

2.As noted in Weakness 1, the construction of negative samples does not appear optimal, as action distance does not equate to a higher task success rate. Could the authors comment on this?

3.From Table 9, I noticed that increasing the number of candidate actions does not lead to sustained performance gains, and the bottleneck is reached very quickly (peaking at K=2K=2, and even dropping at K=8K=8). This is inconsistent with most current test-time scaling works. Could the authors explain why this happens?

4.I am also somewhat puzzled by Figure 5. In Figure 5(a), it appears that the robot resets/retreats because the value growth rate fails to meet expectations, yet the action value immediately after the reset is even lower than before. Could the authors explain this phenomenon?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.