acceptodds
Under review as a conference paper at ICLR 2027

When do discrete actions make good continuous controllers?

Abstract

Though modern AI models like LLMs predict discrete tokens, today's robotic foundation models represent actions as continuous vectors. In this paper, we challenge this common consensus by asking if continuous action parametrizations are superior to their discrete counterparts. We conduct an extensive comparison of single-step regression and multi-step flow variants of both continuous and discrete policies in both from-scratch single-task and multi-task VLA training, in both simulation and on real hardware. In all regimes, we find that discrete action prediction, even the most naive, uniform-bin quantization, can match or even outperform continuous-action policies in task success rate. Moreover, we find that the challenge to discrete action policies is not action precision, but rather factorization error introduced by product-distribution predictions across time and action dimensions. We show that, when paired with the correct interventions—iterative computation through discrete flows, or distribution sharpening through classifier-free guidance—discrete flow policies can turn this weakness into a strength, enabling more responsive action changes during a predicted action chunk. Lastly, we show that attempts to explicitly inject geometric information into discrete policies, e.g. through corruption processes that emulate continuous flow matching, do not yield consistent improvements. Taken together, our findings question existing assumptions that robotic action prediction requires distinct design decisions compared to other AI domains, and motivates the possibility of discrete action prediction as a compelling alternative to today's continuous action parametrizations. Link for the webpage https://anonymous-wolf-submissions.github.io/iclr-submissionhttps://anonymous-wolf-submissions.github.io/iclr-submission

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision Language Action Models, discrete flow matching, robotic, Behavior Cloning

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_EZsvOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Through a systematic empirical study, this paper investigates whether discrete action representations can achieve performance comparable or even superior to continuous action representations in robotic manipulation. The authors compare four policy formulations including CR, CF, DR, and DF, finding that simple uniform quantization suffices to achieve competitive task success rates. Further analysis reveals that discrete policies better capture temporal action variations and state responsiveness, although independent token prediction may introduce factorization errors, while iterative refinement and classifier free guidance help mitigate this issue. Moreover, empirical results indicate that sophisticated tokenization methods and explicit geometric constraints fail to outperform the best uniform quantization scheme. Through experiments on RoboMimic, RoboCasa, LIBERO, and real world robots, combined with analyses of action prediction error, temporal variation, and prediction consistency, the authors validate the effectiveness and potential advantages of discrete action representations.

Critical strengths

None

Critical weaknesses

None

Questions for authors
  1. On RoboCasa, the performance gap between DF (53.9 ± 1.2%) and CF (52.7 ± 3.4%) is relatively small. Meanwhile, the two methods differ substantially in the number of generation steps across several experiments; for instance, in RoboMimic, DF uses 100 steps whereas CF uses only 9 steps. However, the paper lacks a comparison of actual inference time. Could the authors provide the inference latency of different policies on identical hardware and analyze the trade-off between task success rate and computational overhead? If DF incurs significantly higher inference costs, its marginal performance gain over CF may be of limited practical value in real-world robot control.

  2. The authors attribute the primary challenge of discrete policies to Factorization Error and explain the effectiveness of iterative refinement and CFG through the reduction of token-level multimodal ambiguity. However, multimodality within a single token is not directly equivalent to joint inconsistency across temporal steps or action dimensions. Furthermore, as noted in Appendix E.1, varying the NFE also alters the flow time corresponding to the final prediction. Could the authors further clarify how the existing experiments disentangle the effects of improved joint consistency, reduced distribution entropy, and changes in the final prediction flow time? Additionally, Appendix F.1 shows that with frequency (quantile) bins, DR achieves a 49.8% success rate even without CFG, despite its factorized prediction structure remaining unchanged. This suggests that tokenization itself may have a substantial impact on performance. Please explain this phenomenon and further clarify the relative contributions of Factorization Error and tokenization to performance.

  3. The paper validates the effectiveness of discrete action representations on three real-world robot tasks, but the description of the experimental protocol is rather brief, and the results are only presented as a small subplot in Fig. 1(b). Could the authors provide additional details, including the number of evaluation trials per task, the exact success rates, and the source and scale of the training datasets?

  4. The main experiments adopt 512 uniform quantization bins, demonstrating that discrete actions are highly competitive under this setting. However, the relationship between quantization resolution and control precision has not been systematically investigated. We suggest that the authors analyze task success rates and action prediction errors across different numbers of bins, particularly on high-precision tasks such as Tool Hang, to illustrate when quantization error becomes a performance bottleneck.

5 comments
AuthorsOpenReview ↗

Inference latency and the success/compute trade-off

We thank the reviewer for the detailed questions on cost, mechanism, the real-robot protocol, and bin count.

Q1. Inference latency and the success/compute trade-off

On RoboCasa, DF and CF use the same action head (about 370M parameters) on the same backbone, so their cost per step is the same and the comparison comes down to the number of steps. We evaluate the same checkpoint of each head at every NFE (success in %, mean ± SD over five evaluation seeds, 500 episodes each):

HeadCFGNFE 125102050100
DF039.4±2.047.0±2.551.2±2.652.5±3.451.5±1.051.9±2.351.9±1.9
CF034.2±2.445.6±1.351.1±3.351.5±3.451.3±1.651.9±2.450.8±3.6
DF230.2±1.852.4±3.653.1±2.153.6±2.553.2±2.552.7±1.952.3±1.5
CF227.1±1.20.3±0.341.9±1.752.4±2.451.7±2.252.3±1.852.6±3.2

DF does not need more steps than CF. Without guidance, both plateau at NFE 5. With guidance, DF reaches 52.4% already at NFE 2, while CF needs NFE 10 for the same result (0.3% at NFE 2).

We time the same checkpoints on one real RoboCasa observation (one H200, bf16, batch 1, no compilation; median of 20 runs after warm-up). The backbone pass is shared by all heads and takes 23 ms. Action-head time per chunk, in ms:

HeadCFGNFE 125102050100
DF07.912.626.750.197.8239480
CF05.09.422.945.089.4218434
DF212.321.648.593.7187464907
CF29.318.144.387.9176437873

One pass: DR 7.3 ms (CFG 0), 11.7 ms (CFG 2); CR 4.9 ms.

A network pass costs about the same in both heads (4.8 vs. 4.4 ms), and guidance doubles it. Under guidance, DF is 2.5× faster than CF at the same success: DF reaches 52.4% in 45 ms per chunk (23 ms backbone + 22 ms head), while CF needs 111 ms. Guided DR takes 35 ms for 49.2%. All of these finish long before a 24-step chunk ends.

In RoboMimic, 100 steps is our default setting for DF. We re-evaluated DF policies trained with the recipe of the paper at each NFE, with the same 256 episodes for every setting (chunk time at batch 1 on one L40S, fp32, without compilation):

NFE5102050100
Tool Hang success0.790.850.830.840.87
Transport success0.600.650.670.630.66
DF chunk time (ms)5396185446886
CF chunk time (ms)4887173405796

As in RoboCasa, success changes little beyond NFE 10, while the chunk time grows linearly with NFE. One network pass costs about 8–9 ms in both flow heads, and the one-pass heads take about 9 ms per chunk (DR 8.8 ms, CR 8.5 ms). In the revision we change the default for DF in RoboMimic to NFE 9 to match CF.

AuthorsOpenReview ↗

Joint consistency, entropy, flow time; quantile-bin DR

Q2. Joint consistency, entropy, flow time; quantile-bin DR

Joint consistency rather than single tokens. We measure factorization error at the chunk level: the number of ambiguous tokens within one chunk (Fig. 4(b), Fig. 9(a)). We also measure incompatible decoded chunks directly in Push-T. From a symmetric start, the pusher should go around the T block either on the left or on the right, and a chunk that switches between the two plans runs into the block (Fig. 5). In that setting we decode 500 chunks per head and count the chunks whose planned path enters the block:

DR CFG 0DR CFG 2DF NFE 1DF NFE 2DF NFE 5DF NFE 10CF (NFE 10)CR
Incompatible chunks (%)18.412.615.010.23.63.22.88.8

Guidance lowers incompatible chunks for DR from 18.4% to 12.6%, and refinement lowers them for DF from 15.0% at NFE 1 to 3.6% at NFE 5, close to CF (2.8%).

Entropy and joint consistency. These are not separate effects: lower token entropy is how joint consistency improves. A token with a sharp distribution rarely keeps mass on more than one mode, so as entropy falls, fewer tokens in a chunk are ambiguous. A chunk can only mix modes if at least two of its tokens are ambiguous, so fewer ambiguous tokens means a lower chance of factorization error. Formally, for a chunk x=(x1,…,xL)x=(x_1,\dots,x_L), the error of decoding it from per-token marginals is KL(p(x) ∥ ∏ip(xi))=∑iH(xi)−H(x)≤∑iH(xi)\mathrm{KL}\big(p(x)\,\|\,\prod_i p(x_i)\big)=\sum_i H(x_i)-H(x)\le\sum_i H(x_i), which goes to zero as the token entropies do. Refinement lowers token entropy because each step conditions on the tokens already committed. Guidance lowers it by sharpening each token's distribution against the unconditional one. Both reduce the ambiguous tokens per chunk (Fig. 4(b), Fig. 9(a)) and the incompatible chunks in Push-T (table above).

Why guidance matters for uniform bins. With uniform bins, the training actions are spread very unevenly over the bins. On each translation axis, 10–14% of all training values fall in the two bins around zero motion, 11–15 times the average share of the 222 bins in use.

This imbalance shapes the ambiguous tokens. A typical ambiguous token has two modes: a narrow spike at the zero-motion bin, inherited from the frequent zero-motion actions, and a broad peak over the bins of the intended motion. The broad peak can hold more total mass, but it is spread over many bins, so each of its bins is lower than the spike. Argmax picks the single highest bin, so it often lands in the spike. Calling the mode with more total mass the heavier mode, at CFG 0 the argmax lands in the heavier mode for only 44% of ambiguous tokens. Because each token decides on its own, the chunk often combines choices from different modes.

Guidance divides out the unconditional distribution, which carries the zero-motion spike. At CFG 2, the argmax lands in the heavier mode for 61% of ambiguous tokens, fewer chunks mix modes, and uniform-bin DR rises from 27.1% to 49.2%.

Why it matters less for quantile bins. With equal-frequency bins, every bin holds the same share of the training actions, so the background is close to uniform from the start. Ambiguous tokens are about half as common (4.2% of translation tokens, against 7.7% for uniform bins), and the argmax already falls in the heavier mode more often (56% against 44%). Guidance has little left to correct, because ambiguity within the chunk is already low.

Relative contributions. For one-pass argmax decoding, tokenization and guidance act on the same mechanism: a biased background distribution turns token ambiguity into incompatible chunks. Quantile bins remove the bias in the representation, and guidance removes it at decoding time. This is also why quantile bins do not help DF (46.0% vs. 53.9% with flat bins; reply to twnJ-Q3): iterative decoding already removes most of the factorization error.

Final flow time. The sentence in App. E.1 contains a typo: 1−1/NFE1-1/\mathrm{NFE} is the share of committed tokens at the last query, α=1−1/NFE\alpha=1-1/\mathrm{NFE}, not the flow time. The final flow time depends on the scheduler. Our DF uses the cubic scheduler α(t)=t3\alpha(t)=t^3 with steps that are uniform in α\alpha, so the last query is at t=(1−1/NFE)1/3t=(1-1/\mathrm{NFE})^{1/3}: 0.93, 0.97, and 0.98 at NFE 5, 10, and 20. Over this range the final flow time barely changes, while ambiguous tokens per chunk fall from 1.50 to 0.22. The drop in ambiguity therefore does not come from a later final prediction. We correct the sentence in the revision.

AuthorsOpenReview ↗

Real-robot protocol & Bin count and control precision

Q3. Real-robot protocol

We add the protocol and a per-task table to App. A.3. The real-robot experiments use a bimanual setup with two 6-DoF arms and three cameras (one base and two wrist cameras, 224×224). Actions are 14-D: joint targets for both arms relative to the current state, plus absolute gripper commands, predicted in chunks of 24 steps. All four heads start from the π0\pi_0 base weights [Black et al. 2025], are pre-trained on a large in-house multi-task bimanual teleoperation corpus (about 250K episodes over more than 150 tasks, recorded at 30 Hz), and are fine-tuned per task on 50 demonstrations of that task, with the same recipe for every head. Each head is evaluated for 10 trials per task with randomized object placement. A trial counts as a success when every bottle ends fully inside the bin and the bin stays upright (throw bottles), when the cap is fully separated from the pen and both rest on the table (open pen cap), or when the mug ends upright and straight (turn mug up).

TaskDFCFDRCR
Throw bottles into bins9/109/108/106/10
Open pen cap7/107/104/104/10
Turn mug up8/107/105/103/10
Total24/3023/3017/3013/30

With 10 trials per cell the intervals are wide, so we read these results as consistent with the simulation ranking rather than as separate evidence: over the three tasks, the flow heads (DF 24/30, CF 23/30) lead the regression heads (DR 17/30, CR 13/30), and DR matches or beats CR on every task.

Q4. Bin count and control precision

We train DF and DR on Tool Hang and Transport with B∈{64,128,256,512,1024}B\in\{64,128,256,512,1024\} bins in two action spaces (relative and absolute end-effector actions), keeping everything else fixed. Each cell is success rate / EEF error (mm).

Tool Hang

BinsDF relativeDF absoluteDR relativeDR absolute
640.82 / 13.00.38 / 12.20.08 / 16.50.05 / 10.0
1280.82 / 9.40.44 / 9.50.11 / 10.90.08 / 6.6
2560.85 / 7.50.53 / 6.50.12 / 6.60.09 / 7.6
5120.87 / 8.50.53 / 8.70.16 / 9.00.12 / 4.1
10240.83 / 7.70.50 / 7.70.20 / 9.50.15 / 5.9

Transport

BinsDF relativeDF absoluteDR relativeDR absolute
640.50 / 8.10.50 / 8.50.20 / 10.90.16 / 13.7
1280.63 / 5.60.56 / 6.00.40 / 7.70.30 / 8.3
2560.60 / 5.30.56 / 5.80.43 / 5.60.37 / 7.7
5120.66 / 4.90.56 / 6.70.47 / 5.30.40 / 5.9
10240.63 / 5.60.60 / 7.70.48 / 5.20.42 / 6.0

The tables show three trends. (1) On Tool Hang, the number of bins matters more in the absolute action space than in the relative one: for DF, success rises from 0.38 to 0.53 with more bins in the absolute space but stays at 0.82–0.87 in the relative space. (2) It matters more for DR than for DF: DR success grows steadily with the number of bins on both tasks (relative actions: 0.08 to 0.20 on Tool Hang, 0.20 to 0.48 on Transport), while DF changes much less. (3) For DF, success levels off above a moderate number of bins: from at most 256 bins in every setting, so our default of 512 bins lies on this plateau.

The EEF error mostly follows the same pattern as success. It depends less on the number of bins for DF than for DR (e.g. Transport, relative actions: 8.1 to 4.9 mm for DF, 10.9 to 5.2 mm for DR), and it mostly stops decreasing beyond 256 bins. Bin resolution is not what limits DR: with absolute actions on Tool Hang, DR has a lower EEF error than DF at 512 bins (4.1 vs. 8.7 mm) but much lower success (0.12 vs. 0.53). The gap comes from one-pass decoding, not from quantization.

References

  • [Black et al. 2025] K. Black, N. Brown, D. Driess, et al. π0\pi_0: A Vision-Language-Action Flow Model for General Robot Control. RSS 2025.
Reviewer_twnJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper compares continuous regression, continuous flow, discrete regression, and discrete flow for robot action prediction. Experiments span from-scratch RoboMimic policies, VLA-scale simulation, and three real-world tasks. The authors find that simple discrete actions can compete with continuous policies and attribute their performance to stronger responsiveness and reduced factorization error through iterative refinement and classifier-free guidance. They also examine geometric priors and alternative tokenizers.

Critical strengths
  • The comparison covers discrete and continuous action heads, with and without iterative prediction, across RoboMimic, pretrained VLAs, and real-world tasks. This breadth makes the results useful for choosing an action representation and shows that simple discretization remains competitive beyond a single benchmark.

  • The quantized-regression baseline is a useful inclusion: it shows that quantizing targets alone does not reproduce the behavior of categorical prediction.

Critical weaknesses
  • Prior work already examines action-token dependencies and iterative prediction; the paper should explain what its findings add to that understanding.

  • The main concern is whether the experiments establish the proposed factorization-error mechanism. Appendix E.1 counts multimodal distributions for individual tokens, which does not directly measure incompatible combinations in decoded action chunks. Appendix E.3 also reports nearly unchanged success from 5 to 20 refinement steps despite a substantial reduction in ambiguity. The diagnostics show that refinement changes the predictions, but do not establish that reducing factorization error explains the performance gains.

  • The responsiveness analysis measures how much predictions change, rather than whether they change appropriately. Higher jerk or larger action shifts could also reflect jitter or incorrect mode switches.

Questions for authors
  1. What evidence directly measures incompatible decoded action combinations and their reduction under CFG or refinement, beyond multimodality in individual-token predictions?

  2. What evidence separates correct corrective responses from greater output variation, accounting for the shifted prediction windows in the responsiveness diagnostic?

  3. Scope of the tokenizer conclusion. How much of FAST’s deficit reflects invalid decoding rather than representation quality?

3 comments
AuthorsOpenReview ↗

Prior work & factorization-error mechanism

We thank the reviewer for the careful reading and the precise questions on the mechanism.

W1. Prior work

We discuss the closest prior work and add a related-work section (App. H).

Token dependencies. FAST [Pertsch et al. 2025] studies a different problem from ours. With per-step bins at high control rates, each token adds almost no information beyond the previous ones, so autoregressive training receives little learning signal. On a toy task the model learns to copy the first action, and on 20 Hz and 50 Hz robot tasks it makes no progress. This is a training problem of autoregressive decoding. The factorization error we study arises at inference in parallel decoding: a one-pass head predicts each token's marginal, and independent choices can combine into a chunk that no demonstration contains. Work on diffusion language models studies this error (ReDi [Yoo et al. 2025], CoDD [Li et al. 2026], and Tang et al. [2026]). To our knowledge, we are the first to measure it in robot action chunks and to test what removes it.

What this adds. (i) The two problems need different remedies. FAST's problem does not arise with parallel decoding: plain per-step bins decoded in one pass beat continuous regression, and flat bins give our best DF, so compression is not needed for parallel heads. (ii) What remains is factorization error, and iterative decoding is not the only fix: guidance in a single decoding step recovers most of the gap (27.1% → 49.2%, vs. 53.9% for DF). Prior discrete VLAs report that iterative decoding beats one-pass decoding without testing why.

W2. Does the evidence establish the factorization-error mechanism?

We do not only count ambiguity in single tokens. Factorization error refers to a chunk that contains more than one ambiguous token: parallel decoding picks each of them independently, so the choices can combine into a chunk that is not feasible. Our diagnostic therefore counts total ambiguous tokens within each chunk: Fig. 4(b) shows these counts with and without guidance, and Fig. 9(a) shows the share of chunks with at least kk ambiguous tokens.

We agree that refinement beyond NFE 5 does not improve success. Adding the low-NFE range shows a clear trend (DF on RoboCasa, CFG 0, mean ± SD over five evaluation seeds; a chunk has 39 translation tokens, three coordinates at 13 time steps):

NFE1251020
Ambiguous tokens per chunk (of 39)3.624.941.500.570.22
Success (%)39.4±2.041.6±2.251.2±2.652.5±3.451.5±1.0

At NFE 1 and 2, success is limited not only by ambiguity but also by prediction precision: with one or two steps, the predicted actions are less precise. This is why success still rises slightly from NFE 1 to 2 (39.4% to 41.6%), although ambiguity peaks at NFE 2. From there the trend is clear: ambiguous tokens per chunk fall from 4.94 to 1.50 at NFE 5, and success rises from 41.6% to 51.2%. Fig. 9(a) shows where this change comes from: the share of chunks with several ambiguous tokens drops sharply from NFE 2 to 5. Beyond NFE 5, ambiguity keeps falling but is already low. A single ambiguous token cannot form an incompatible combination on its own, so little factorization error is left to remove, and success stays flat (51.2–52.5% up to NFE 100).

Direct measurement on decoded chunks (Q1). As the reviewer suggests, we also measure incompatible combinations directly, in Push-T, where incompatible chunks have a very clear pattern. From a symmetric start, the pusher should go around the T block either on the left or on the right. A chunk that switches between the two plans runs into the block (Fig. 5). On the exact setting of Fig. 5, we decode 500 chunks per head and count the chunks whose planned path enters the block:

DR CFG 0DR CFG 2DF NFE 1DF NFE 2DF NFE 5DF NFE 10CF (NFE 10)CR
Incompatible chunks (%)18.412.615.010.23.63.22.88.8

Both remedies reduce incompatible chunks, as the factorization account predicts. Guidance lowers DR from 18.4% to 12.6%. Refinement lowers DF from 15.0% at NFE 1 to 3.6% at NFE 5, close to CF (2.8%), and adds little beyond that (3.2% at NFE 10). This matches RoboCasa, where success rises up to NFE 5 and then stays flat. CR, which predicts the mean of the two plans, also runs into the block in 8.8% of chunks. These counts measure incompatible combinations in the chunks the policy executes, not multimodality of single tokens.

AuthorsOpenReview ↗

Responsiveness: size of change vs. correct change

W3. Responsiveness: size of change vs. correct change

We agree that the size of a change alone cannot separate a correct change from jitter. The chunk change is a vector, so we now also compare its direction with the demonstrated change, and add this to App. D.3.3.

Take two frames o1,o2o_1, o_2 of the same held-out demonstration, kk steps apart. Let a⋆(o)\mathbf{a}^\star(o) be the demonstrated chunk at frame oo, μπ(o)\mu_\pi(o) policy π\pi's mean predicted chunk, and eπ(o)=μπ(o)−a⋆(o)\mathbf{e}_\pi(o)=\mu_\pi(o)-\mathbf{a}^\star(o) its error. These four chunks give

Δμπ=Δa⋆+eπ(o2)−eπ(o1):\Delta\mu_\pi=\Delta\mathbf{a}^\star+\mathbf{e}_\pi(o_2)-\mathbf{e}_\pi(o_1):

the policy's change is the demonstrated change plus the change in its error. We project the policy's change onto the demonstrated change:

ρπ∥=⟨Δμπ,Δa⋆⟩∣Δa⋆∣2,ρπ⊥=∣Δμπ−ρπ∥Δa⋆∣∣Δa⋆∣.\rho^{\parallel}_\pi=\frac{\langle\Delta\mu_\pi,\Delta\mathbf{a}^\star\rangle}{\\|\Delta\mathbf{a}^\star\\|^2},\qquad \rho^{\perp}_\pi=\frac{\\|\Delta\mu_\pi-\rho^{\parallel}_\pi\Delta\mathbf{a}^\star\\|}{\\|\Delta\mathbf{a}^\star\\|}.

For each pair, ∣Δμπ∣2/∣Δa⋆∣2=(ρπ∥)2+(ρπ⊥)2\\|\Delta\mu_\pi\\|^2/\\|\Delta\mathbf{a}^\star\\|^2=(\rho^{\parallel}_\pi)^2+(\rho^{\perp}_\pi)^2, so the size ratio ρπ\rho_\pi behind Fig. 3 and Table 7 splits into a part along the demonstrated change and a part across it. ρπ∥=1\rho^{\parallel}_\pi=1 means the policy changes as much as the demonstration in the demonstrated direction; ρπ∥<1\rho^{\parallel}_\pi < 1 under-responds. ρπ⊥\rho^{\perp}_\pi is change that the demonstration does not make, which jitter or a wrong mode switch would raise. Both chunk changes are taken over the same two windows, kk steps apart, so the window shift appears on both sides. App. D.3.3 of the revised paper illustrates this decomposition (Fig. R1).

We use the 1,483 frame pairs of Table 7 (normalized units, CFG 0, NFE 10, 32 draws per frame), which reproduce its medians. All four heads receive random inputs (App. B.6), so for every head the floor of ρπ⊥\rho^{\perp}_\pi applies the same projection to the difference between two independent estimates of μπ(o1)\mu_\pi(o_1). ρπ\rho_\pi is the size ratio from Table 7; ρπ⊥\rho^{\perp}_\pi is given with its floor in parentheses:

HeadLag 1: ρπ\rho_\piρπ∥\rho^{\parallel}_\piρπ⊥\rho^{\perp}_\piLag 4: ρπ\rho_\piρπ∥\rho^{\parallel}_\piρπ⊥\rho^{\perp}_\piLag 16: ρπ\rho_\piρπ∥\rho^{\parallel}_\piρπ⊥\rho^{\perp}_\pi
DF0.680.1040.68 (0.55)0.700.2380.63 (0.20)0.820.5800.56 (0.09)
CF0.550.0550.52 (0.31)0.630.1320.53 (0.11)0.760.4210.54 (0.05)
DR1.080.0561.06 (0.08)0.940.1370.80 (0.03)0.930.4730.68 (0.01)
CR0.530.0400.46 (0.05)0.580.1110.46 (0.02)0.720.3960.50 (0.01)

The split shows three things. First, DF has the largest parallel part at every lag (0.10, 0.24, and 0.58 at lags 1, 4, and 16), so its change follows the demonstration more closely than any other head's. Second, along the demonstrated direction, DR is close to CF and above CR. Third, the reviewer's concern partly holds for DR: its perpendicular part is larger than those of the continuous heads. All heads move only part of the way along the demonstrated change (ρπ∥<0.6\rho^{\parallel}_\pi < 0.6 at lag 16).

Within each pair, the discrete head follows the demonstrated change more than its continuous counterpart, at every lag: DF over CF (0.10 vs. 0.06, 0.24 vs. 0.13, 0.58 vs. 0.42) and DR over CR (0.06 vs. 0.04, 0.14 vs. 0.11, 0.47 vs. 0.40). The direction test therefore supports the responsiveness claim of Sec. 3 with the direction of the change, not only its size.

W3 (continued)

Jitter within a chunk. The same concern applies to the jerk ratio of Eq. (2). With the third finite difference of the translation commands, Δ3ahtr=ah+3tr−3ah+2tr+3ah+1tr−ahtr\Delta^3\mathbf{a}_h^{\mathrm{tr}}=\mathbf{a}_{h+3}^{\mathrm{tr}}-3\mathbf{a}_{h+2}^{\mathrm{tr}}+3\mathbf{a}_{h+1}^{\mathrm{tr}}-\mathbf{a}_h^{\mathrm{tr}} for h=1,…,H−3h=1,\dots,H-3, it compares only the sizes of the predicted and demonstrated jerk:

Rjerk=1H−3∑h=1H−3∣Δ3a^htr∣1H−3∑h=1H−3∣Δ3ahtr∣.R_{\mathrm{jerk}}=\frac{\frac{1}{H-3}\sum_{h=1}^{H-3}\\|\Delta^3\hat{\mathbf{a}}_h^{\mathrm{tr}}\\|}{\frac{1}{H-3}\sum_{h=1}^{H-3}\\|\Delta^3\mathbf{a}_h^{\mathrm{tr}}\\|}.

Jerk at the wrong steps or in the wrong directions can still give Rjerk≈1R_{\mathrm{jerk}}\approx1. We therefore also report the relative jerk error, which compares the jerk vectors step by step:

Ejerk=∑h=1H−3∣Δ3a^htr−Δ3ahtr∣∑h=1H−3∣Δ3ahtr∣.E_{\mathrm{jerk}}=\frac{\sum_{h=1}^{H-3}\\|\Delta^3\hat{\mathbf{a}}_h^{\mathrm{tr}}-\Delta^3\mathbf{a}_h^{\mathrm{tr}}\\|}{\sum_{h=1}^{H-3}\\|\Delta^3\mathbf{a}_h^{\mathrm{tr}}\\|}.

Ejerk=0E_{\mathrm{jerk}}=0 when the predicted jerk equals the demonstrated jerk at every step, and Ejerk=1E_{\mathrm{jerk}}=1 for a prediction with no jerk (Δ3a^htr=0\Delta^3\hat{\mathbf{a}}_h^{\mathrm{tr}}=\mathbf{0}). So Ejerk<1E_{\mathrm{jerk}} < 1 means the predicted jerk follows the demonstrated jerk better than no jerk at all, and Ejerk>1E_{\mathrm{jerk}} > 1 means it adds error. By the triangle inequality, Ejerk≥∣Rjerk−1∣E_{\mathrm{jerk}}\ge|R_{\mathrm{jerk}}-1|: matching the size of the jerk is necessary but not sufficient. We compute both on the chunks of Fig. 2.

HeadRjerkR_{\mathrm{jerk}}EjerkE_{\mathrm{jerk}}
DF0.50±0.070.80±0.05
CF0.37±0.111.09±0.04
DR0.72±0.300.96±0.23
CR0.10±0.081.01±0.02

For each source we take the median over its 1,000 chunks (CFG 0, NFE 10; chunks with zero target jerk are excluded, as in Fig. 2), and report the mean ± SD of these medians over the eight sources. The results show that discrete formulation not only produces jerk closer in size to the demonstrations, but also matches the demonstrated motion direction better than their continuous counterparts.

Q1. Direct evidence of incompatible decoded combinations

We address this together with W2 (Part 1 of our response).

Q2. Correct corrections vs. more output variation

We address this together with W3 (Parts 2 and 3 of our response).

Q3. FAST: invalid decoding vs. representation quality

For the parallel heads, most of it. FAST is designed for autoregressive decoding, and its sequence grammar cannot be met by one-pass independent prediction. FAST DR produces 0.5% valid decodes through training and has no closed-loop result; FAST DF produces 25–40% valid decodes. With autoregressive decoding, FAST decodes validly almost always and reaches 48.2%.

The other tokenizers separate the two causes. nFSQ, RVQ, and quantile bins always decode to valid actions, yet DF with them reaches 52.4%, 38.2%, and 46.0%, below flat bins (53.9%). For these tokenizers the gap reflects how hard the tokens are to predict, not invalid decoding.

To measure FAST's share directly, we resample invalid FAST DF chunks until they are valid, instead of falling back to a hold action: 48.9% vs. 46.2% with the hold fallback. The small remaining gap to flat bins is expected: FAST produces variable-length token sequences, while a parallel head predicts a fixed number of tokens. FAST does not suit parallel decoding heads; we do not claim its representation is worse.

References

  • [Pertsch et al. 2025] K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, S. Levine. FAST: Efficient Action Tokenization for Vision-Language-Action Models. arXiv:2501.09747, 2025.
  • [Yoo et al. 2025] J. Yoo, W. Kim, S. Hong. ReDi: Rectified Discrete Flow. NeurIPS 2025.
  • [Li et al. 2026] I. Li, Z. Shao, B. Wang, R. Yu, G. Van den Broeck, A. Liu. Breaking the Factorization Barrier in Diffusion Language Models (CoDD). ICML 2026.
  • [Tang et al. 2026] L. Tang, L. Yu, S. Zhang, G. Ver Steeg. Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Models. ICML 2026.

Comments on the paper

2 comments
AuthorsOpenReview ↗

scope, quantization error, and single-step policies

We thank the reviewer for the careful reading and the questions on scope.

W1. Claims broader than the tested scope

Yes, we entirely agree that humanoids, dexterous hands, and higher-frequency "System 0" controllers, including torque control, are very interesting directions for future study. Given the length, scope, and experimental detail of the present submission, we felt that extending to these domains with the same rigor would be too much for a single submission. We are eager to explore them in future work.

Within manipulation, the study already covers a wide range of settings: single-task policies trained from scratch on RoboMimic, multi-task VLA-scale policies on RoboCasa, DROID, and LIBERO, and three real-robot tasks, with relative and absolute end-effector actions and relative joint actions. We state this scope explicitly in the discussion of the revision.

Q1. Torque, dexterous hands, locomotion, high frequency

We do not expect quantization error to become the bottleneck. Each action coordinate is normalized before binning, so the bins cover that coordinate's own range. At a higher control rate, per-step actions are smaller, but so is their normalized range, and the bin width shrinks with them. A single-step policy therefore has the same relative quantization error as the first step of a chunk. Our bin sweep on Tool Hang and Transport agrees (reply to Reviewer EZsv, Q4): DF success levels off by 256 bins.

What can remain is factorization error across coordinates. Even at a single time step, a policy for a dexterous hand or a humanoid predicts many coordinates, and a one-pass head chooses each of them independently. We expect this part to grow with the number of coordinates, so iterative decoding or guidance should matter more in these settings.

Prior work also suggests that quantization is rarely the bottleneck. Optimal control with bounded inputs often has bang-bang solutions, which use only the extreme values of each input [Bellman et al. 1956; LaSalle 1960]. In reinforcement learning, per-dimension action bins work well on continuous-control benchmarks, including locomotion [Tavakoli et al. 2018; Tang & Agrawal 2020], and bang-bang policies with two values per coordinate are competitive [Seyde et al. 2021]. Dactyl controls a dexterous hand with 11 bins per coordinate [OpenAI et al. 2020].

Q2. Tied to action chunking?

Factorization error across time steps exists only when a policy predicts a chunk, so a single-step policy removes that part of the error. The part across action coordinates and the responsiveness to observations remain. Single-step discrete policies such as RT-1 already work well [Brohan et al. 2022].

To test this directly, we train all four heads as single-step policies (H=1H=1) on Tool Hang and Transport and evaluate each for 100 trials (success rate):

TaskDFCFDRCR
Tool Hang0.550.500.100.06
Transport0.490.430.370.28

Without chunking, success drops sharply for every head, but the ranking matches the chunked results: DR stays above CR on both tasks, and DF and CF perform similarly. The advantage of discrete heads therefore does not come from action chunking alone.

References

  • [Bellman et al. 1956] R. Bellman, I. Glicksberg, O. Gross. On the "Bang-Bang" Control Problem. Quarterly of Applied Mathematics 14(1):11–18, 1956.
  • [LaSalle 1960] J. P. LaSalle. The Time Optimal Control Problem. In Contributions to the Theory of Nonlinear Oscillations, Vol. V, pp. 1–24. Princeton University Press, 1960.
  • [Tavakoli et al. 2018] A. Tavakoli, F. Pardo, P. Kormushev. Action Branching Architectures for Deep Reinforcement Learning. AAAI 2018.
  • [Tang & Agrawal 2020] Y. Tang, S. Agrawal. Discretizing Continuous Action Space for On-Policy Optimization. AAAI 2020.
  • [Seyde et al. 2021] T. Seyde, I. Gilitschenski, W. Schwarting, B. Stellato, M. Riedmiller, M. Wulfmeier, D. Rus. Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies. NeurIPS 2021.
  • [OpenAI et al. 2020] OpenAI, M. Andrychowicz, B. Baker, et al. Learning Dexterous In-Hand Manipulation. The International Journal of Robotics Research 39(1):3–20, 2020.
  • [Brohan et al. 2022] A. Brohan, N. Brown, J. Carbajal, et al. RT-1: Robotics Transformer for Real-World Control at Scale. arXiv:2212.06817, 2022.
Reviewer_9Rv4OpenReview ↗

Official Comment by Reviewer 9Rv4

Thanks for the authors' response. Most of my concerns have been addressed. I decide to keep my original positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.