acceptodds
Under review as a conference paper at ICLR 2027

Evaluating Test-Time Scaling in Robot Policies and World Models

Abstract

Vision-Language-Action (VLA) models and World Models (WMs) are becoming increasingly capable tools for robot decision-making, enabling action generation and prediction of future outcomes. These capabilities make sampling additional actions and their imagined futures a natural approach to scaling computation at test time. We introduce a diagnostic benchmark of sample-based test-time scaling that asks when additional samples improve action selection, and how well WM imagined futures can help with selecting the right action. Our evaluation spans VLAs, integrated World-Action Models (WAMs), VLAs augmented with action-conditioned WMs, and GPT-6 Astra with a waypoint scaffold. We organize the core metrics into three dimensions: sampling potential, measured by what is the best achievable gain from sampled actions assuming a perfect oracle selector is available (oracle headroom); selection effectiveness, measured by selector gains over random selection on the same candidate pool; and uncertainty informativeness, measured by how well dispersion in sampled actions predicts failure. Our results show additional samples expose substantial oracle headroom, reaching nearly 50 percent for Astra with a waypoint scaffold on LIBERO, but tested selectors fail to achieve the same headroom consistently. We also diagnose what is behind this action selection failure and discuss future directions for WM research.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotics foundation model, world model, VLA, benchmark, test time scaling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WrztOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper examines whether sampling more actions at test time improves policy performance and whether world models help select successful actions. The authors introduce a diagnostic benchmark across several policy families and simulation environments. They measure the potential gain from perfect action selection, the actual gain achieved by practical selectors, and whether dispersion among sampled actions predicts failure. They evaluate action consensus, value-based selection, and VLM judgments of predicted futures, using simulator-generated futures to diagnose selection failures. First, they find potential gains in some settings, but practical selectors do not consistently realize them. Second, longer ground-truth rollouts improve ranking in some cases, while longer predicted rollouts do not reproduce this benefit in their Motus experiment. Finally, they find that the usefulness of action dispersion for failure prediction varies by setting and execution stage.

Critical strengths

None

Critical weaknesses
  • The contribution of this paper is vague. I am unsure exactly what the authors want to claim, because the key takeaways of Sections 3, 4, and 5 do not lead to a clear overarching conclusion. They largely show that sampling potential, selection effectiveness, and uncertainty informativeness depend on the specific experimental setup. This dependence is unsurprising, so the new generalizable insight remains unclear.
  • The selection baselines are too narrow, and the proposed future directions do not adequately consider prior work. The authors already discuss several selection approaches beyond VLM-based judging in the related work, including a relative action critic (VLA-ATTC), policy-derived selection signals (MG-Select), and geometric consistency (GeoBoN), but do not evaluate these approaches at all. Simply calling for stronger native value heads or explicitly trained external verifiers is therefore not a sufficiently novel future direction.
Questions for authors

Q1. Can the authors identify any consistent trend across the evaluated settings in selection effectiveness (Section 4) or uncertainty informativeness (Section 5)? The current takeaways seem unsurprising, and it is unclear whether the positive findings extend beyond the specific experimental setups.

Q2. Can the authors evaluate the rollout-refined Cosmos Policy planning model mentioned in Section 4.2 on the proposed benchmark, or explain why it cannot be included? This seems an important reference for value-based selection, since the current experiment only uses native values from the action-proposal checkpoint.

Q3. Can the proposed benchmark be extended to any real-world robot evaluations to test whether its takeaways hold outside simulation?

Reviewer_yL9vOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a diagnostic benchmark for sample-based test-time scaling in robot policies and world models. The evaluation separates three aspects: the sampling opportunity measured by oracle headroom, the effectiveness of practical candidate selectors, and the usefulness of action or predicted-future dispersion for failure prediction. Experiments cover six policy systems across four simulator families. The main findings are that additional samples expose substantial oracle headroom, the tested selectors do not consistently recover this potential, and failure-prediction signals depend on the policy and execution stage. Further controls using simulator-generated images and privileged states investigate the roles of prediction accuracy, evaluation, and lookahead horizon.

Critical strengths

A systematic evaluation that separates sampling opportunity from selection quality.

The strongest aspect of the paper is its evaluation design. Measuring oracle headroom and practical selection gain on the same candidate pools helps distinguish the availability of successful candidates from the ability to identify them. The simulator-image and privileged-state controls in Section 4.3 further probe whether selection remains difficult when prediction errors or image interpretation are partially removed. Applying this framework across multiple policy families and simulator environments provides a useful empirical reference for evaluating test-time sampling. This systematic collection of measurements is valuable even when the tested selectors do not yield consistent improvements.

Critical weaknesses
  1. The central findings offer limited new insight beyond existing understanding of sampling and candidate selection.

The first headline finding—that additional samples can expose substantial oracle headroom—is largely expected from best-of-N evaluation. For a fixed nested candidate pool, oracle success cannot decrease as more candidates are added, although the empirical gap relative to random selection need not increase monotonically. The magnitude of this gap and its variation across systems are useful measurements, but the existence of sampling opportunity is not, by itself, a particularly novel scientific insight.

The distinction between generating a successful action and identifying it has also been studied in prior work. For example, SVA, which is cited in Section 2, already uses a pass@k diagnostic to demonstrate successful behaviors within a frozen VLA’s output distribution and then trains an action evaluator from search results. VLA-ATTC and RoboMonkey likewise explicitly address candidate selection through trained critics or verifiers.

Against this background, the paper does not sufficiently establish what substantially new understanding emerges from its broader evaluation. The short-horizon versus long-horizon controls are more informative than the basic oracle-headroom observation, but the results do not yet yield a clear, generalizable explanation of when or why additional computation improves action selection. Overall, the novelty appears to lie primarily in experimental coverage and organization, while the incremental scientific insight is less convincing.

  1. The proposed implications remain unvalidated, limiting the practical value of the diagnosis.

Section 6 recommends more reliable long-horizon prediction, representations suitable for task-value prediction, and stronger candidate evaluators. However, these recommendations remain research directions: the paper does not implement and validate a targeted improvement showing that addressing an identified limitation actually increases selection gain.

I do not consider a new algorithm mandatory for a diagnostic benchmark. My concern is that, given the limited novelty of the central observations, the paper needs stronger evidence that its diagnosis leads to an actionable understanding. A focused intervention demonstrating improved selection would substantially strengthen that contribution.

The current controls leave some important ambiguity. Longer simulator futures may reveal task completion and therefore make retrospective outcome recognition easier, as acknowledged in Appendix C.2. The recursive Motus experiment combines predicted-image feedback with approximate proprioception, so its negative result does not isolate which source of error limits performance. Moreover, the evaluated selectors exclude value heads trained on imagined futures. Consequently, the experiments establish limitations of the tested configurations more clearly than they establish which concrete change would reliably overcome those limitations.

Questions for authors

What is the most consequential, non-obvious conclusion established by this benchmark beyond the already-studied existence of sampling headroom and the difficulty of candidate selection? Please identify the specific experiment that supports this distinction from prior work such as SVA and VLA-ATTC.

Why are trained candidate-selection methods, such as the relative action critic in VLA-ATTC or a learned verifier, not included in the experimental comparisons? A comparison on compatible tasks and matched candidate pools would help determine whether the observed gap primarily reflects limitations of the evaluated selectors.

Can one of the proposed remedies be validated through a focused experiment? For example, does a task-conditioned value head trained on world-model features improve held-out selection gain under the same sampling budget? Alternatively, can a stronger control isolate predicted-image error from approximate proprioception in the recursive Motus experiment? Either would make the diagnosis more actionable.

Reviewer_3bcrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies sample-based test-time scaling for robot policies. At a decision state it samples N candidate action chunks, executes each one from the same reset or replayed state followed by the policy's closed-loop continuation, and records binary success. It reports oracle headroom, defined as realised best-of-N success minus the pool mean (Eq. 1); the gain of practical selectors over random selection on the same pool; and how well dispersion among sampled actions predicts failure (AUROC). The study covers six systems—GPT-6 Astra with a waypoint scaffold, Motus, LingBot-VA, Cosmos Policy, GR00T-N1.7 and π0.5—and four simulator families: LIBERO (including LIBERO-Plus), RoboCasa, RoboTwin and MolmoSpaces. Headroom at N = 8 ranges from 5.9 to 49.7 pp (Table 1). Action consensus, VLM judges of imagined futures and Cosmos Policy's native value head do not establish gains over random (Table 3). Simulator-rendered long-horizon futures improve ranking for Motus and GR00T, but recursive Motus imagination does not. Action dispersion predicts failure strongly for Cosmos Policy on LIBERO-Plus (AUROC up to 0.89) and weakly elsewhere.

Critical strengths
  1. A clean decomposition for evaluating claims about test-time scaling. Oracle headroom and selection gain are measured on the same candidate pool, with matched pools and paired intervals. A privileged-information ladder separates errors of the predictor from errors of the judge. The ladder goes from imagined image, to simulator image, to simulator state, at increasing horizons. Why it matters: test-time-scaling papers usually report only end-to-end gains. This protocol shows where the gains are lost.

  2. Breadth and candour. Six systems spanning VLAs, world-action models, modular VLA + world model pairings and a frontier model. Cluster-bootstrap intervals and within-task permutation tests. A five-seed replication of the Cosmos Policy failure-prediction result (Table 18), and a failed replication that is reported honestly (RoboCasa value mean, App. D.4). The negative results are useful to the field: Action consensus gains 0.5 pp even though 19 of 22 LingBot pools have a successful majority. Cosmos Policy's native value gives −1.40 pp despite +6.93 pp of oracle headroom (Table 3A).

Critical weaknesses
  1. The headline headroom may be mostly outcome noise rather than differences a selector could exploit. How the oracle is computed. Each candidate is executed once, followed by a stochastic closed-loop continuation. O_N takes the maximum of these single outcomes. Why that cannot tell noise from signal. Candidates are i.i.d. draws. If continuation noise is independent across executions, the single outcomes at a root are i.i.d. Bernoulli with that root's mean success p̄. So O_N = E[1 − (1 − p̄)^N] depends only on root-level mean success. It would be identical whether success varies with the candidate or only with the continuation (derived). Example (derived). Re-executing one fixed candidate eight times, with success probability 0.416 (Astra/LIBERO's pool mean, Table 5), already gives 1 − 0.584^8 − 0.416 ≈ 57 pp of "headroom". Variation in success across roots lowers this figure; the observed value is 49.7 pp. The point is not that all of the headroom is noise. It is that this metric cannot separate noise from differences a selector could exploit. What the replay controls cover (Table 8): Deterministic re-execution: Motus (0/100 flips) and Cosmos Policy on LIBERO-Plus (0/160). Noisy: LingBot-VA (9/96 flips) and GR00T (25/160). Astra has no flip-rate control. Its continuations are stochastic, and its API "does not honor sampling seeds" (App. B). Yet Astra's 49.7 pp is the abstract's headline. The deterministic settings still overstate what a deployed selector can get. Outcomes there are tied to one continuation seed; RoboTwin, for example, uses common random numbers. The oracle therefore selects with knowledge of future noise that a deployed selector would not have. Label noise also limits the judge metric. The same noise caps pairwise accuracy, so near-chance judge results cannot be interpreted until it is quantified. Why this matters: the paper's central quantity and its main message ("selectors fail to capture the headroom") both depend on how much of the headroom is selectable.

  2. It is not yet a standardised, reusable benchmark, and the reporting is incomplete. Different protocol per setting. Settings differ in tasks, seeds, N (4 vs 8), decision points and horizons. Many selector × system combinations are not run. Action consensus appears in the main results only for Motus and LingBot-VA (Table 3A). An exploratory Astra row in Table 9 does not report N or episode counts. LingBot-VA's imagined futures are not judged (Table 13, "Unmeasured"). Published selectors are not run. The trained verifiers and selectors discussed in Section 2 (RoboMonkey, MG-Select, VLA-ATTC, SVA) are not evaluated. The VLM judges use a fixed 0–100 "candidate-approach" rubric (two prompt configurations in Table 10). On the Cosmos Policy/RoboCasa pool, the Gemma judge "frequently gives all candidates tied scores" (Section 4.2). Reporting gaps. By my count, about 20 places in the appendix say a count, interval, setting or control is "unreported", "unavailable" or "unspecified". Examples: rows of Table 9, Table 13 ("Pointwise CIs are unavailable"), the Astra judge temperature, and the initially missing API responses. No release commitment. The reproducibility statement does not commit to releasing the harness, candidate pools or outcome labels. The anonymous project page says "Code on acceptance". Several headline settings depend on a proprietary hosted model. Why this matters: a benchmark paper's value lies in a fixed, reusable protocol and complete reporting. The authors themselves note that "differences in cohorts, evaluators, and compute constrain cross-model comparisons" (App. C, Scope and limitations).

Questions for authors
  1. For each setting in Table 1, can you report a selectable oracle? Either re-execute each candidate under several independent continuation seeds, or use a split-half oracle: pick the candidate that succeeded in execution 1 and score it in execution 2. Did the same-candidate controls in Table 8 vary the continuation seed, or replay it?

  2. For Astra, what fraction of the 49.7 pp survives re-executing the selected candidates?

  3. Will the harness, candidate pools, outcome labels and replay states be released? Which settings form the fixed core benchmark (N, tasks, selectors)?

  4. Can you add at least one trained verifier or published selector on the same pools? Examples: a likelihood or confidence score in the style of MG-Select, or a value head trained on candidate outcomes. Can you also add a learned head on imagined-future features, the direction Section 6 proposes?

  5. At t = 128–256, how often is the task outcome already visible in the image or state? The authors note such images "may reveal task completion". Can the long-horizon ranking be restricted to roots whose outcome is not yet decided?

  6. Two results are significantly inverted: the GR00T state classifier at t = 8 (0.22) and Gemma at t = 256 (0.276). Could leave-one-root-out bias with small per-task training sets explain the first? Could the "candidate-approach" rubric explain the second? Was the permutation test applied to the full leave-one-out pipeline?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.