acceptodds
Under review as a conference paper at ICLR 2027

Select to Succeed: Pareto-Consistent Action Selection for VLAs via Augmented Tchebycheff Scalarization

Abstract

Scaling test-time computation in Vision-Language-Action (VLA) models necessitates robust arbitration over stochastically generated candidate trajectories. We formalize test-time candidate selection as a finite-set multi-objective optimization problem under a receding-horizon control framework, introducing S2S-VLA (Select to Succeed) for frozen VLA policies. Specifically, we map candidate action prefixes into a predefined cost space comprising the marginal probabilities of task failure and local anomalies, alongside an analytic metric of command roughness. To systematically resolve objective trade-offs, we construct the selection rule via an augmented Tchebycheff scalarization and prove that its exact minimizers are nondominated within the evaluated candidate set in the predicted objective space. To align a lightweight neural evaluator with this selection criterion, we optimize a theoretically compatible dual-objective formulation: a pointwise binary cross-entropy loss anchoring marginal event probabilities to empirical offline branch rollouts, and a pairwise logistic loss enforcing scalarization-induced soft pairwise preferences. Following temperature scaling for post-hoc probability calibration, S2S-VLA deterministically minimizes the scalarized score, enabling mathematically consistent, rollout-free, and computationally efficient test-time scaling for embodied agents. Extensive experiments in simulated and real-world environments demonstrate substantial improvements in task success over baseline methods. Our project is available at S2S-VLA.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA models, Multi-objective optimization, Pareto consistency, Augmented Tchebycheff scalarization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_i58EOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper addresses test-time scaling for frozen VLAs: the policy samples several candidate action chunks and must pick one to execute. Existing Best-of-N approaches rank candidates with a single learned scalar score. The authors argue that such a score hides trade-offs between objectives and tends to favor candidates whose success is overestimated. They propose S2S-VLA, which scores each candidate on three costs: predicted task-failure probability, predicted anomaly probability, and an analytic command-roughness term. The candidate is chosen by minimizing an augmented Tchebycheff scalarization, which guarantees that the choice is Pareto-nondominated within the candidate set. A lightweight evaluator on frozen VLA features predicts the two probabilities. It is trained offline on simulator branch rollouts, with a pointwise BCE loss plus a pairwise ranking loss, and then calibrated with temperature scaling. Applied to OpenVLA, π0 and π0.5, S2S-VLA consistently improves success on RoboCasa, LIBERO-Plus and RLBench, by about 7–8 points on average. It also helps in a small real-robot study.

Critical strengths
  • The method plugs into frozen VLAs, and gives consistent gains across three backbones and three benchmarks at a moderate latency cost.
Critical weaknesses
  • There is no comparison with other test-time scaling methods, and no compute-matched trade-off. Every result compares a backbone only with itself plus S2S. Several established methods sample action candidates from a VLA and select among them, and they are much simpler:

    • BID (Liu et al., 2025) and MG-Select (Jang et al., 2026) need no extra verifier.
    • RoboMonkey (Kwok et al., 2025) uses Gaussian perturbation, majority voting and a VLM verifier.
    • V-GPS (Nakamoto et al., 2024) re-ranks candidates with a value function.
    • TACO (Yang et al., 2025) uses a pseudo-count verifier.
    • CoVer-VLA (Kwok et al., 2026) also learns a verifier.

    None of these is evaluated, and BID, MG-Select and TACO are not even cited. The paper reports latency against the number of candidates for OpenVLA only. It never puts success against total cost next to these alternatives. The total cost includes online latency for every backbone and the offline cost of supervision. Without this comparison, it is unclear whether S2S-VLA beats a simpler selector at the same compute.

  • The pipeline is complex, and neither its components nor its scalability is justified. S2S-VLA requires all of the following:

    • a resettable simulator with 15 branch rollouts per candidate, which is 240 rollouts per training context at 16 candidates
    • task-specific anomaly definitions
    • a separate evaluator for each backbone
    • temperature calibration
    • a preference search on validation episodes.

    None of these choices is ablated. The paper never tests Tchebycheff against a weighted sum, and never tests the pairwise loss, temperature scaling, the roughness term or the preference search individually. The only ablation removes objectives. The two propositions are standard or hold by construction, so they do not justify the design either. The paper also never discusses how the cost grows with new tasks or embodiments. It does not explain how the real-robot evaluator was supervised, given that branch rollouts are impossible there.

Questions for authors
  • How does S2S-VLA compare with BID, MG-Select, RoboMonkey, V-GPS, TACO and CoVer-VLA, using the same backbone and the same number of candidates?
  • Please report success against total compute for S2S-VLA and these baselines. This should include online latency for all three backbones and the offline cost of collecting branch rollouts and training the evaluator.
  • Can you ablate each component individually: Tchebycheff versus a weighted sum, the pairwise loss, temperature scaling, the roughness term and the preference search? Please also report calibration error.
  • What concrete advantage over simpler or training-free selectors justifies this pipeline? How does its cost grow with new tasks, embodiments and backbones? Does each new task need new branch rollouts and anomaly definitions?
  • How was the evaluator trained for the real-robot experiments without branch rollouts? How does cross-backbone transfer (Figure 7) work when the evaluator’s input features are backbone-specific?
1 comment
AuthorsOpenReview ↗

Response to Reviewer i58E

Dear Reviewer i58E,

Thank you for recognizing our method's consistent gains across backbones and benchmarks at a moderate latency cost.

Matched candidate count. On RoboCasa at K = 16, S2S gained 1.4–2.0 percentage points over BID across backbones and 1.0 point over MG-Select on OpenVLA. Relative to CoVer-VLA, the differences were +0.5, −0.1, and −0.4 points on OpenVLA, π0, and π0.5, respectively.

MethodOpenVLAπ0π0.5
Base policy, K = 133.6%42.3%48.1%
BID39.0%49.0%54.0%
MG-Select40.0%N/AN/A
RoboMonkey40.0%50.0%55.0%
V-GPS39.0%49.0%54.5%
TACO39.5%50.0%55.0%
CoVer-VLA40.5%50.5%56.0%
S2S41.0%50.4%55.6%

MG-Select uses action-token distributions, which OpenVLA supplies. The two π checkpoints use continuous action heads.

Inference time and execution quality. On OpenVLA, K = 16 gave 41.0% success at 0.35 s per step. Increasing K to 64 gave 41.9% at 1.30 s (Figs. 5a and 8b). For π0 and π0.5, each applicable baseline could tune K to match S2S's end-to-end decision time on one GPU.

Measured quantityπ0π0.5
Time per decision0.32 s0.36 s
Best baseline success at that budget50.5%55.5%
S2S success50.4%55.6%

On π0.5 at 0.36 s, S2S recorded two fewer anomalous prefixes per hundred than the highest-success baseline (9% vs. 11%) and lower mean normalized roughness (0.42 vs. 0.46), with a 0.1-point difference in success.

Offline and deployment cost. BID and MG-Select require no separate verifier fit. V-GPS, TACO, RoboMonkey, and CoVer-VLA use learned scorers. For S2S, 1,000 contexts × 16 candidates × 15 repeats require 240,000 branches. At 2,000 branches per GPU-hour, collection takes 120 GPU-hours. Fitting adds 8 GPU-hours and calibration plus preference validation adds 2, for 130 GPU-hours of setup. At 0.36 s each on π0.5, 10,000 sequential decisions correspond to about one GPU-hour of online decision time. Eight repeats save 46.7% of branches and give 55.2% success versus 55.6% with 15.

Component ablations. Full S2S reached 55.6% on RoboCasa with π0.5. Changing one component gave 55.0% with a tuned weighted sum, 54.8% with probability-only training, 55.5% with unit calibration temperatures, 54.3% with roughness removed, and 55.0% with fixed equal weights. Success-only ranking reached 50.8% (Fig. 5b). On held-out in-domain RoboCasa data, temperature scaling halved expected calibration error, from 0.08 to 0.04 for success and from 0.06 to 0.03 for anomalies.

Proposition 1 guarantees nondominance within the valid set under the predicted costs. Proposition 2 shows that fitting the empirical probabilities exactly also reproduces the pairwise targets. Both losses reuse the same labels. At deployment the evaluator scores all valid prefixes in one batch.

Transfer and scaling. In Fig. 7a, the target policy proposes the prefixes. The frozen source encoder reads the observation and instruction for its own evaluator head, adding one encoder pass per off-diagonal decision. On π0.5 proposals, the π0 evaluator reaches 53.8%, recovering 5.7 of the matched evaluator's 7.5 points above the 48.1% base policy. New tasks can reuse the detector and evaluator when event definitions and candidate rankings carry over. New embodiments also need compatible action coordinates. Changed event definitions or poor target-domain ranking can require fresh branches and refitting.

PiPER supervision. We trained the evaluator on simulator branch outcomes and froze it for hardware evaluation. We used the 50 physical demonstrations per category to adapt the proposal policy. In Fig. 8a, the base policies completed 81/180 trials (45.0%) and S2S completed 109/180 (60.6%) across three backbones and three categories.

Reviewer_eY6DOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes S2S-VLA, a test-time scaling framework for frozen Vision-Language-Action (VLA) policies that formulates candidate action arbitration as a finite-set multi-objective optimization problem under receding-horizon control.Experiments on RoboCasa, RLBench, LIBERO-Plus, and real robots show consistent improvements (≈7–8 percentage points on average) over single-sample baselines.

Critical strengths
  1. Casting VLA candidate selection as an explicit multi-objective problem with a principled scalarization (augmented Tchebycheff) is a clean and compelling alternative to single noisy scores.
  2. The approach is rollout-free online and backbone-agnostic, making it attractive as a drop-in selection module for many VLA systems.
  3. The analytic command roughness metric provides a simple, architecture-agnostic regularizer that does not require model capacity for kinematics.
Critical weaknesses
  1. Cost constraints. Offline supervision cost could be substantial (NKR branch rollouts with per-candidate branching).
  2. The continuation policy π_ref used for supervision differs from the closed-loop policy induced by repeated reselection, potentially causing systematic bias in the success-probability target.
  3. The local anomaly definition is task-specific and not operationally standardized. And the paper does not provide detailed detection/labeling protocols or show robustness to alternative anomaly definitions.
Questions for authors
  1. How are local anomalies defined and labeled across tasks, especially in real robots? Are the definitions automatic (sensor-based thresholds) or manual?
  2. Given the supervision uses πref for continuation, did you observe systematic biases when the closed-loop controller repeatedly re-selects? Any evidence that modeling p_s under re-selection (e.g., via off-policy simulation) would materially change results?
1 comment
AuthorsOpenReview ↗

Response to Reviewer eY6D

Dear Reviewer eY6D,

Thank you for highlighting the efficiency of our rollout-free selector at inference and its applicability across VLA backbones.

Does the continuation policy change the decision? The success head estimates task completion when the frozen base policy takes over after a candidate prefix. On the same prefixes from held-out states visited by S2S, the mean success probability was 0.52 under the reference continuation and 0.56 under repeated S2S selection. The two continuations agreed on the ordering of 86% of within-state candidate pairs. At the same branch count, refitting on S2S continuations changed closed-loop success by 0.2 percentage points, from 55.6% to 55.8% on fresh RoboCasa episodes with π0.5.

How are anomalies labeled? In simulation, a task-specific detector automatically marks a branch if a qualifying event occurs during its executed prefix. Contacts required by the instruction are allowed, and an episode can still succeed after a recoverable event. We applied two definitions to the same RoboCasa executions. Each entry is the fraction of executed prefixes with at least one event.

Event definitionBase π0.5S2S
Object drop or displacement of a non-target object8%6%
Broader rule also counting unintended contact12%9%

For the PiPER evaluator, both heads were trained on simulator branch labels. We used the 50 physical demonstrations per category to adapt the proposal policy and froze both models for hardware evaluation. Figure 8a reports instruction completion over 20 trials per policy and category. The prefix-level anomaly rates in the table come from the simulator's detector.

How much branch supervision is needed? With 16 candidates per training context on RoboCasa, we varied the number of branch rollouts for each candidate and measured π0.5 task success.

Repeats per candidateBranch rollouts per contextTask success
46454.0%
812855.2%
1524055.6%

Reducing the repeats from 15 to 8 saved 46.7% of branch rollouts, with a 0.4-point drop in observed success. Each branch supplies both event labels, which are also used to form the pairwise preference targets. Testing another preference reuses the evaluator and requires only the additional validation episodes.

Reviewer_Akb1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

S2S-VLA improves frozen VLA policies by selecting among multiple sampled action prefixes. A learned evaluator predicts task-success and local-anomaly probabilities, which are combined with an analytic command-roughness measure through augmented Tchebycheff scalarization. The evaluator is trained using offline branch rollouts and complementary probability and preference objectives. Experiments report improvements across three VLA backbones, several simulation benchmarks, and real-world manipulation tasks.

Critical strengths
  1. The framework makes the trade-offs involved in action selection explicit. Its separation of predicted event probabilities and analytic command roughness gives the selection objective a clear interpretation, and the theoretical claims carefully specify the candidate-set and predicted-objective setting.

  2. Experiments cover multiple policy backbones and both simulated and physical environments. Candidate-count and objective-composition ablations provide useful evidence about the performance–latency trade-off and the contribution of the auxiliary objectives.

Critical weaknesses
  1. Comparisons with alternative action-chunk execution strategies are insufficient. The main results primarily compare the original policies with their S2S-equipped versions. They do not establish the advantage over adaptive action-chunk execution, which changes when to replan or how much of a chunk to execute, or mixture/aggregation strategies that combine action proposals. Although these approaches intervene at different points in the controller, they are relevant alternatives for improving temporal consistency and execution reliability. Could the authors compare representative alternatives under matched backbones, available supervision, and inference budgets, including whether S2S provides additional benefits when combined with adaptive chunk execution?

  2. The necessity of the particular scalarization and training formulation is unclear. The objective-composition ablation establishes the usefulness of considering multiple costs, but does not isolate augmented Tchebycheff scalarization from simpler alternatives. Positive weighted sums also yield nondominated exact minimizers over a finite candidate set. Comparisons using the same evaluator with weighted-sum selection, together with probability-only training and removal of calibration, would help identify which components account for the observed improvement.

  3. The empirical evidence for probability calibration and risk reduction is limited. Calibration and anomaly avoidance are central motivations, yet the reported evaluation emphasizes task success. Candidate-set nondominance in predicted costs does not establish accurate probabilities or lower realized anomaly rates, particularly after repeated reselection changes the state distribution. Could the authors report held-out calibration metrics, executed anomaly rates, and command roughness before and after selection, including under distribution shift?

Questions for authors

See Weakness

1 comment
AuthorsOpenReview ↗

Response to Reviewer Akb1

Dear Reviewer Akb1,

Thank you for recognizing the clarity of our multi-objective approach to action selection and the breadth of our evaluation.

Prefix selection and adaptive execution. On RoboCasa with π0.5, both selectors used 16 candidates per decision and the same branch supervision. In the adaptive setting, the scheduler chose the execution length first. Each selector scored the valid prefixes of that length, and the robot executed the selected one.

Selection ruleFixed horizonAdaptive horizon
Predicted success alone50.8%52.0%
All three S2S objectives55.6%56.0%

With the same adaptive scheduler, using all three objectives raised success from 52.0% to 56.0%. For the full selector, switching from fixed to adaptive execution added 0.4 percentage points. At comparable inference time per episode, the standalone adaptive controller reached 52.5% and temporal ensembling of overlapping chunks reached 53.5%.

Scalarization and training. We agree that positive weighted sums also yield nondominated minimizers. In the example in Appendix B.2, every positive weighted sum favors at least one endpoint, (0, 1, 0) or (1, 0, 0), over the middle candidate (0.6, 0.6, 0). With equal weights and augmentation 0.05, our rule selects the middle candidate, which scores 0.22 versus 0.35 for either endpoint. On the same calibrated evaluator, a validation-tuned weighted sum gave 55.0% success and augmented Tchebycheff gave 55.6%. Training on the same branches with the probability loss alone gave 54.8%. Setting both calibration temperatures to one gave 55.5%. The larger gain came from adding the anomaly and roughness objectives to success-only ranking (50.8% to 55.6%).

Calibration and executed outcomes. Each cell reports held-out expected calibration error (ECE) or Brier score before and after temperature scaling. We evaluated the success head against its frozen-policy continuation target. The transferred evaluator retained its source-domain temperatures.

Metric on RoboCasaTrained on RoboCasaTrained on RLBench
Success ECE0.08 → 0.040.13 → 0.10
Anomaly ECE0.06 → 0.030.10 → 0.08
Success Brier0.22 → 0.210.25 → 0.24
Anomaly Brier0.10 → 0.090.14 → 0.13

On held-out states visited by S2S, continuing the same prefixes gave mean success probabilities of 0.52 with the frozen base policy and 0.56 with repeated S2S selection. To measure executed outcomes, we counted prefixes with at least one observed anomaly on the same RoboCasa tasks. We report mean command roughness from Eq. 5 on a common normalized scale across controllers.

π0.5 controllerTask successPrefix event rateMean roughness
Base policy48.1%12%0.50
Best latency-matched baseline55.5%11%0.46
S2S, evaluator trained on RoboCasa55.6%9%0.42
S2S, evaluator trained on RLBench38.5%13%0.47

At matched latency, the RoboCasa-trained evaluator yielded two fewer anomalous prefixes per hundred than the highest-success baseline at a 0.1-point difference in task success. After RLBench-to-RoboCasa transfer, calibration improved with temperature scaling, while the 13% anomaly rate exceeded the base policy's 12%.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.