acceptodds
Under review as a conference paper at ICLR 2027

Candidate–Priority Coupling in Discrete-Diffusion Robot Policies

Abstract

Successful robot execution does not reveal whether a policy can recover multiple valid behaviors from the same state. We identify a source of coverage loss in discrete-diffusion decoding: ranking token positions by the probability of the sampled candidates that will be committed. An exact selection identity shows how this candidate–priority coupling can distort the output distribution even with exact conditionals. Sample- split confidence ranks positions with an independent pilot draw and commits a separate draw, removing this dependence. On ForkBench-Choice, a controlled manipulation task, matched two-draw interventions in the first policy query hold weights and model-call budgets fixed. Sample splitting improves primary-fit Coverage@10 by 20.4 percentage points (95% paired-state bootstrap interval: 11.1–29.6), while satisfying a prespecified validity guard. A second fitted checkpoint shows a consistent gain, with disclosed technical qualifications. A separately trained VLA also shows increased diversity on a fresh Approach-Side panel when choosing between prevalidated fixed-controller routes. Together, the exact identity and matched no- remasking interventions identify candidate–priority coupling as a source of finite-budget behavioral coverage loss. Decoding rules determine which valid behaviors a fixed policy recovers and warrant evaluation alongside task success.

Primary area:
applications to robotics, autonomy, planning
Keywords:
discrete diffusion, vision-language-action models, robot learning, candidate–priority coupling, behavioral diversity, decoding, sampling bias

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_nhcHOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates whether, in discrete diffusion robot policies, determining the token submission order based on the probabilities of already-sampled candidate values suppresses the coverage of effective behavior modes.

Critical strengths

The paper is difficult to read and follow, which makes it hard to assess its strengths. The writing is often unclear and does not meet the standard of clarity expected for a conference submission. I encourage the authors to revise the manuscript substantially, particularly its motivation and presentation.

Critical weaknesses

The paper is very difficult to read and follow. The figures do not clearly convey the paper’s central motivation, and their design makes the main message hard to understand. The research question and the motivation for the proposed method also need clearer explanations. The connection between the theoretical analysis and the proposed method is unclear. The paper does not include a table of experimental results, please don't use figure. In addition, there is no real-robot demo.

Questions for authors

I recommend that the authors substantially improve the paper’s readability.

1 comment
AuthorsOpenReview ↗

Response to Reviewer

Thank you for pointing out how hard the main result was to find. Take the instruction “put either can in the basket.” A robot that always picks soup could succeed every time. If we start it from the same state ten times, does it ever pick tomato?

We changed one decision in the decoder. It predicts several action tokens at once and must choose which positions to fill first. With Coupled, a position gets its score from the very token the decoder will write there. With Split, a separate draw from the same predictions supplies that score. The theorem shows how the first rule can favor some sampled values. The table shows what happened when we switched rules in the same trained policy.

Primary fit, ten attempts per stateCoupledSplitRandomMax
States with at least one success (/27)25252525
States where we saw both cans placed across separate attempts (/27)4151217
Valid rollouts (/270)185197187189

We prepared a revision that opens with this example and places the numerical comparison at the start of Results. All of the Choice rollouts were closed-loop simulations. We have no physical-robot demonstration.

Reviewer_ykwzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a decoding rule to rank token positions using the probabilities of the sampled values. The authors derive an exact selection identity showing how this dependence can distort the distribution even with exact conditions. The sample-split intervention ranks positions using an independent draw, while committing a separate candidate. Matched experiments demonstrate increased valid-mode coverage on a manipulation task, and a separately trained policy also produces more diverse choices.

Critical strengths

Matched intervention isolates a meaningful decoder dependence. Split and Coupled consume two draws and 12 model forwards, with matched mask counts and position noise. Coverage@10 increases to 0.741 from 0.537 with low p-value.

Critical weaknesses
  1. Theoretical contribution does not predict the learned decoder's behavioral effect.
  • Theorem 1 follows by conditioning on the selected coordinate and value; the preservation result assumes exact conditional sampling and sequential commitment.
  • However, the policy commits multiple tokens from learned distributions.
  • It supplies neither a preservation guarantee nor a quantitative explanation for the robot decoder.
  1. The experiments show a narrow improvement, but do not establish broader usefulness for robot control.
  • The main results come from one manipulation task, where both policies come from the same 27 starting states. The second task tests choosing between scripted routes. Although the descriptive comparisons do not invalidate the matched Split-Coupled result, they do limit the practical significance -- whether removing the decoder dependence yields a better robot policy.
Questions for authors

See weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer

We agree about the limit of Theorem 1. It shows that choosing which token to keep can change the output distribution even with exact conditionals and a single token committed at a time. The 20.4 percentage point gain on the robot task has to be measured.

One part of the argument carries over to the trained parallel decoder. Fix the visible history just before a decoding step. The pilot and the candidate are independent draws from the same per-position distributions. Ranking either draw therefore gives the same marginal distribution over selected blocks. With Split, knowing that a position was chosen tells us nothing further about the value that will be written there. With Coupled, that value helped determine whether the position was chosen.

We checked this at the first denoising step, where the logits and both candidate arrays matched across conditions. For each newly filled position, we took the log probability of the committed value and subtracted the expected log probability of a draw at that position. Coupled increased this prespecified score over Split by .144 on the primary fit (95% paired-state interval [.067, .234]) and .075 on the qualified second fit ([.012, .176]). So we can see the preference for higher-probability values in the tokens the learned decoder actually kept. We cannot say how much of the eventual coverage difference this first-step effect explains.

On task success, the answer is straightforward. Both decoders succeeded in the same 25 of 27 states within ten attempts. They used two draws per step and 12 model forwards in the first query. The per-rollout validity difference was .044 [−.007, .096], so we cannot conclude that validity was equivalent. What improved was the chance to see both valid behaviors within the available attempts: we observed both in 15 states with Split and 4 with Coupled, a paired Coverage@10 gain of .204 [.111, .296].

On 54 fresh Approach-Side contexts, a separately trained policy chose between two prevalidated controller routes. The fraction of pairs of proposals that chose different sides rose from .245 to .378 (paired gain .133 [.094, .173]). This gives us evidence about route choice in another policy. We still lack a second task on which the learned policy itself successfully executes the actions in closed loop.

Thank you for the detailed rebuttal. The results that have been added I think still raises my second point: max confidence and random ordering do as well as Split, or better. I think my other concerns are mostly resolved, but I do not think I will be raising my score.

Reviewer_PgkiOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how discrete-diffusion decoding affects the recovery of multiple valid robot behaviors. It shows that using a sampled token’s probability to determine its own commitment priority can distort the output distribution, even under exact conditional predictions. An independent pilot sample removes this dependence in the analyzed sequential setting. Controlled robot experiments compare coupled and split sampling with fixed policy weights and matched decoding budgets, demonstrating improved finite-budget behavioral coverage.

Critical strengths
  1. The paper identifies a specific selection mechanism and connects an exact mathematical result to a carefully matched intervention. Holding policy weights, candidate draws, and model-call budgets fixed provides convincing evidence that decoding can affect coverage for the evaluated checkpoints.
  2. The evaluation distinguishes task success from valid-mode coverage and reports uncertainty, validity checks, negative controls, and limitations. This makes the empirical claims more interpretable than reporting diversity without accounting for execution validity.
Critical weaknesses
  1. The interaction between decoding effects and the quality of language-conditioned policy training remains unresolved. The fixed-checkpoint intervention demonstrates a decoding effect, and the exact analysis shows that such an effect need not originate from poor training. However, the magnitude and practical relevance of the observed coverage loss may still depend on the small task-specific adaptation dataset and the learned strength of each behavior mode. The teacher-forcing and expert-prefix diagnostics are helpful, but do not replace a training-by-decoder ablation. Could the authors compare stronger or more extensively trained language-conditioned policies under the same decoders, and test explicit instructions requesting each individual mode? Successful instruction following is distinct from diverse sampling under an instruction allowing either mode, but measuring both would clarify how policy training and decoding contribute to the observed limitation.

  2. The closed-loop evidence remains narrow. The principal result concerns one controlled two-mode task and a fixed 27-state panel. The second checkpoint has disclosed technical qualifications, while Approach-Side evaluates choices between prevalidated fixed-controller routes rather than new closed-loop policy executions. These studies support the local mechanism, but provide limited evidence about its prevalence across tasks and independently trained policies. An additional closed-loop task with a clean independent training run would materially strengthen the empirical scope.

  3. The practical benefit of sample splitting over simpler candidate-independent priorities is not established. Max-confidence achieves higher coverage than Split on the primary checkpoint, and random ordering does so on the recovered checkpoint. The paper appropriately presents Split as a mechanistic intervention, but the resulting recommendation for decoder design remains incomplete. Under what conditions should practitioners prefer splitting over these simpler alternatives, and are there meaningful differences in validity or coverage across sampling budgets?

Questions for authors

See Weakness

1 comment
AuthorsOpenReview ↗

Response to Reviewer Pgki

We agree that the amount of training could matter. The Choice policy saw just ten demonstrations per object, all under an instruction that allowed either can. What we can measure with those weights held fixed is how often each decoder recovers both choices. Over ten attempts per starting state, both decoders succeeded at least once in the same 25 of 27 states. In 4 states with Coupled and 15 with Split, we saw the robot put soup in the basket on one attempt and tomato on another. All 11 gains came from recovering the second behavior.

We haven’t tested a more extensively trained policy or asked the robot to pick a named can. The latter would answer a question our either-can task leaves open: can the policy choose tomato when the instruction specifically asks for it? We do have a second training run with the same schedule (seed 17). Split again improved Coverage@10, by 13.0 percentage points (95% paired-state interval, 5.6 to 22.2). That run failed its original training-completion check and reused the same 27 states, so we report it as a qualified sensitivity check.

We also tried closed-loop execution on Approach-Side with a separately trained policy. It had zero successes in 120 development attempts. In the subsequent experiment, the policy chose between two routes that a fixed controller had already executed successfully. Choice therefore remains our only task with successful closed-loop policy executions.

We would use Split when we want to know whether a sampled token helps determine its own chance of being kept. At a given decoding history, an independent pilot preserves the marginal distribution over which positions get filled. Random and Max change that distribution as well. Both had higher observed coverage than Coupled on both fits. At ten attempts Max had the highest coverage on the primary fit (.778, versus .741 for Split), and Random had the highest on seed 17 (.852, versus .759 for Split). These were descriptive comparisons.

To answer your budget question, we averaged coverage over every pair and every group of five among the ten recorded attempts per state, including failures. On the primary fit, Coupled and Split reached .409 and .499 at two attempts, then .487 and .660 at five. Max reached .482 and .659. Across all ten attempts, the valid-rollout fractions were .685 for Coupled, .730 for Split, .693 for Random and .700 for Max. The smaller-budget estimates are a post-hoc analysis of those same rollouts. We prespecified Coverage@10 for the matched comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.