acceptodds
Under review as a conference paper at ICLR 2027

When can test-time search be skipped? Successor-Channel Criticality for Action-Chunked Policies

Abstract

Test-time search can improve action-chunked robot policies, but running it at every decision is expensive and often does not change the outcome. We ask when search can be skipped. Our answer uses the policy's own candidate actions: if every candidate leads to a state with a similar capacity for future control, search has little to choose between. We measure successor-channel criticality as the variance across candidate actions of a future-controllability score. The score summarizes the spread of object positions reached by subsequent policy rollouts at a fixed resolution, and is computed in a resettable simulator without task reward or search labels. We prove that low criticality bounds the value of search, provided that value differences the score does not capture are also small. We evaluate gates with paired per-state interventions on fourteen datasets from three benchmark suites and two policy classes; criticality's ranking lies above the matched random band on all fourteen. Skipping search where estimated criticality is zero avoids about half of decisions while giving up 19.9% of the net benefit, less than the mean of the same score or a cheap rule based on episode time and sampled gripper disagreement. Distilled into a small network and added to that rule, a criticality head raises success on 600 held-out episodes by 4.2 points with 20.5% fewer search calls on a LIBERO image policy, and by 7.0 points at similar call counts on Stack. A network trained on the mean score also improves on that rule; on the image policy, averaged over three head seeds, the criticality head reaches 2.2 points higher success at similar calls when search is scarce and similar success with fewer calls at .

Primary area:
applications to robotics, autonomy, planning
Keywords:
Test-Time Search, Search Allocation, Action-Chunked Policies, Robot Learning, Causal Evaluation, Value of Computation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_78LgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

During a sequence of actions, a robot needs to make a decision at every node about which actions to take. The previous approach is to first simulate and see what is likely to happen after each action. But this is equivalent to exhaustive search, which leads to very large time and compute costs. The authors propose a method that uses the difference in these future possibilities to judge whether it is worth investing more resources in search. They compare with some other methods for deciding whether to search, and the results are reasonably good.

Critical strengths
  1. The problem they focus on is important. When a robot handles a task through a sequence of actions, search is indeed needed to achieve better results, but the cost of search is high. They use a pruning-like approach to reduce the scope of search. 2. The methods they compare with are fairly sufficient and fairly numerous.
Critical weaknesses
  1. Their main tests are all conducted in simulation environments, which makes them rather dependent on the simulation environment. Could validation be done in multiple simulation environments at the same time? What I mean is testing with different physics engines. Because each simulator may have its own deviation from the real world, and this deviation may lead to wrong results regarding actual success or failure. 2. This idea is not completely novel. Moreover, regarding how these evaluation values are computed (why these evaluation values are computed this way), I would like to see more ablations.
Questions for authors

Same as Weaknesses.

Reviewer_9zw5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses successor-channel criticality as variance of reachability for future states to determine of test time search will be useful for action chunked policies. Across various datasets and different policies, criticality ranks better than random. However, comparisons with simpler signals and mean score are mixed. In experiments on a LIBERO vision policy, adding a distilled criticality head to a simpler gate improves success with fewer or similar search calls.

Critical strengths

Broad set of experiments. The evaluation protocol is a useful contribution with paired interventions with and without search estimate search gains at each state instead of estimating from episode success. This reveals that common gating signals can switch direction across datasets and even after retraining the policy. This can become useful for adaptive-compute VLAs independent of the proposed criticality gating which has weaker support.

Critical weaknesses

1 The method relies on privileged access at every stage. The teacher requires a resettable simulator, object world-state, and many rollouts per state. Search also uses a privileged evaluator. These limitations are acknowledged but the path to observation-only deployment or real hardware is missing.

2 The benefit over simple baselines is limited. Offline comparisons with cheap signals and the mean score are mixed. In the closed loop setting, the cheap gate accounts for most of the gain.

3 Writing clarity could be improved. The paper is dense and the relationship between the reachability, criticality estimation, theoretical bound, and deployed gate becomes less clear. Improving this would help but 1/2 are the key issues.

Questions for authors

How much of the deployment gain remains when only observations are available to the policy? Can a learned model replace simulator resets? Does criticality still help with a non-privileged evaluator, eg a learned critic? The evaluations currently suggest that the choice matters considerably. Can more tasks and in particular seeds be added to check whether criticality consistently improves on mean-score?

Reviewer_qUNWOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to address when test-time search over candidate action chunks can be skipped for action-chunked robot policies. Its answer is successor-channel criticality: the variance across candidate first chunks of a future-controllability score, computed reward-free from policy rollouts in a resettable simulator with a split-half estimator that removes Monte-Carlo noise inflation. Theorem 5.1 decomposes action-value variance into a criticality term plus a residual, so low criticality plus a small residual bounds the value of search. A second contribution is a causal evaluation protocol: paired single-switch interventions with common random numbers measure each state's search benefit against cross-fitted oracle and random references. This yields a notable negative result — ten of fourteen common gating signals flip direction across corpora, one even across policy retraining. Criticality ranks above the random band on all fourteen corpora, and a zero-criticality veto skips 49.4% of decisions while giving up 19.9% of net benefit, beating mean (25.8%), level (33.4%), and cheap-gate (23.5%) controls. Distilled into a small network (still reading privileged state) and combined with a cheap gate, the criticality head gains 4.2 success points with 20.5% fewer calls on a LIBERO image policy and 7.0 points at similar calls on Stack. However, a successor-mean head also improves over the cheap gate, and the criticality-vs-mean edge is small and often unresolved. Evaluation is confined to MuJoCo simulation with a fixed privileged best-of-eight operator. The authors are unusually transparent about limitations, including veto-set instability and sensitivity to the benefit measure.

Critical strengths

S1. Rigorous evaluation. Paired single-switch interventions with common random numbers and a cross-fitted oracle measure the per-state causal value of a fixed search operator, a reusable methodological contribution. It produces the important negative result that 10/14 common signals flip direction across corpora (one across policy retraining), which alone justifies the paper.

S2. The mean-vs-variance question. The successor mean uses the same rollouts as criticality, so the comparison tests the statistic rather than the pipeline, and the split-half estimator fixes a genuine noise-inflation pitfall. The signal beats the random band on 14/14 corpora, and the zero-criticality veto skips ~49% of decisions while giving up less net benefit than all controls.

S3. Extensive experiments. Pilot operating points are selected on call counts, deployment tests use 600 paired held-out resets with head-seed replication and random-target controls, and unresolved comparisons and protocol deviations are reported explicitly. The claims are appropriately calibrated and the artifacts auditable.

Critical weaknesses

W1. Privileged-information dependence undermines practical significance. The teacher, deployed head, and search evaluator all read simulator state and success labels; nothing is shown from pixels, with a learned evaluator, or on hardware, so the method is not yet usable in the deployments that motivate it.

W2. The theory is not operational. The bound depends on κ\kappa and a residual the paper itself shows cannot be bounded reward-free, so Corollary 5.2 certifies no actual state; meanwhile the deployed veto is a clamped point estimate with no significance control, and 31.6% of Stack states flip membership across estimator seeds.

W3. The variance-vs-mean advantage is weakly supported where it matters. With cheap cues included, the criticality and successor-mean heads are nearly tied offline and mostly unresolved closed-loop; the veto conclusion is also measure-sensitive (under positive-part benefit the cheap gate gives up less: 22.7% vs 30.3%).

Questions for authors

Q1. Under positive-part benefit the cheap gate beats the veto (22.7% vs 30.3%). Which measure should govern the veto claim, and does the 19.9% vs 23.5% ordering survive label-noise-robust analysis?

Q2. The image-policy headline compares gates whose realized call counts drifted from the pilot match (2.26 vs 2.84). Can you re-threshold the cheap gate on the test resets to isolate the success effect at matched spend?

Q3. The privileged evaluator ties at ~84.5% of Stack decisions. How much measured ΔS\Delta_S variation is genuine criticality versus evaluator inability to rank candidates?

Q4. Can hθh_\theta be trained from pixels or the frozen policy's visual embeddings? Even a small pilot would substantially address Weakness 1.

Comments on the paper

1 comment

Dear reviewers,

Thank you for your work reviewing this paper!

The authors have submitted their responses. If you haven't already, please acknowledge them in a comment and adjust your reviews as needed. The discussion period finishes on November 18.

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.