acceptodds
Under review as a conference paper at ICLR 2027

Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?

Abstract

Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective. However, as we show through extensive analysis, sufficiently accurate action reconstruction is only one part of what makes a downstream robot policy successful. It is also critical that the policy is able to predict the right tokens for new observations, and that unseen policy token predictions still decode into reasonable actions. These properties are downstream of tokenizer training and are not directly incentivized by a reconstruction objective alone. In this work, we introduce Predictable and Robust Action Tokenization (ProAct), a tokenizer training method that strategically augments reconstruction with the goal of improving downstream predictability and robustness. ProAct is policy-agnostic and uses only action datasets for training. Across the Robomimic, LIBERO, and RoboTwin benchmarks and a diverse set of tokenizer architectures, ProAct improves rollout success by an average of 11.3 percentage points. These improvements also translate to vision-language-action policies and real-world robotic manipulation, yielding average gains of 21.8 and 36.7 percentage points, respectively. These results suggest that effective action tokenization should be designed as a policy interface—balancing fidelity, predictability, and robustness—rather than as a reconstruction problem alone.

Primary area:
applications to robotics, autonomy, planning
Keywords:
action tokenization, imitation learning, autoregressive policies, discrete representation learning, robot manipulation, action chunking

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Hr8pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies learned action tokenizer for autoregressive robot policies and evaluates over 22 different tokenizer configurations. The benchmarks used for evaluation are Robomimic, LIBERO, RoboTwin and real robot experiments, plus a VLA experiment on LIBERO. The study reveals that, beyond a sufficient level of accuracy, reconstruction error for action chunks does not predict how successful a policy is during rollouts and it separates matched token predictions from mismatches. The paper introduces an improved method for action tokenizer called ProAct. ProAct uses token balance regularization and token perturbation to improve results of most evaluated tokenizers on the three different benchmarks without needing environment observations or other robot related data beside the action chunks.

Critical strengths
  • Broad evaluation across many different benchmarks, configurations and models
  • Easy to follow and well motivated paper
Critical weaknesses
  • The framing doesn't match the evaluation setup. The work is motivated through autoregressive VLMs and VLAs, yet all evaluations use a small 4-layer, d=256 transformer trained from scratch, which is closer to BeT, VQ-BeT and QueST than to a VLA. The main architecture is not disclosed in the main text.
  • The VLA does not predict action tokens with the VLM itself: a separate, randomly initialized 43M Llama action expert predicts the 8 tokens from a 128-token action vocabulary, using only the VLM’s keys and values (Appendix Table 12). This differs from the ‘native next-token interface of autoregressive VLMs’ used to motivate action tokenization in the introduction.
  • Both ProAct components mainly build on existing methods. TP adapts RobustTok and TBR is an existing preconditioner. Clarifications regarding novelty beyond motivation and application would strengthen the contribution.
  • TBR mechanism is only shown for a linear encoder
  • The reported 48.1% on LIBERO is far below published results. We acknowledge the smaller model and limited training budget, but this makes it unclear whether ProAct’s gains persist for competitive VLAs.
  • It is not clear if the models trained and evaluated are a multi-seed evaluation or only using one seed. If only one seed was used the claims are weak as it is not known if other models are within variance/noise -The overall gain of using either TBR or TP on robomimic is rather low. Showing this ablation on the other benchmarks would help to show that it actually matters or analyzing more in what context ProAct actually improves action tokenization.
  • 4.1 is not a new discovery and well known in the field of robot learning. It would be helpful to explain why this is new in the context of action tokenization and autoregressive policies to make it more clear.
  • Neither FAST nor BEAST was evaluated as a baseline.
  • Paper is generally well written, but key details are only in the appendix (Policy details, VLA details)
Questions for authors
  • Tokenizer reconstruction error is not reliable to tell if the policy is successful (line 61). But is this not clear? It is the same as looking at the validation loss and saying the policy is good because the loss is low. This is all in open loop and does not say anything about performance.
  • Line 66-67 how do the token sequences differ from the targets and why is this not controlled through the training objective?
  • Could you explain what exactly the novelty of the work is and what parts already exist in their form and which are newly proposed?
9 comments

Thank you for the detailed review.

1. "The framing doesn't match the evaluation setup. … yet all evaluations use a small 4-layer, d=256 transformer trained from scratch …"

The paper is about autoregressive policies, including VLAs. The main sweep uses a compact autoregressive policy (Table 10) because it is a controlled study: the policy recipe is fixed and only the tokenizer changes. A study of 22 configurations across 3 benchmarks with VLAs is beyond our compute budget. Not all of our evaluations use the compact policy, however. In the VLA experiment, ProAct improves 7 of 8 tokenizer pairs, from 26.2% to 48.1% average success (Sec. 6.2, Table 2), which suggests that the findings of the controlled study carry over.

2. "The main architecture is not disclosed in the main text." / "Paper is generally well written, but key details are only in the appendix …"

Thank you for finding the paper well written. The revision allows one additional page, so we will move the main policy and VLA architecture and training details into Sec. 6.1 and keep the full tables in App. B.3 (Tables 10 and 12).

3. "Both ProAct components mainly build on existing methods. TP adapts RobustTok and TBR is an existing preconditioner. …" / "… what parts already exist in their form and which are newly proposed?"

We agree, and we will separate the two more clearly in the paper. What already exists: TBR uses the activation-preconditioning component of DoPr (App. D.2), and the TP sampler is adapted from RobustTok (App. D.3); we will credit both more clearly in Sec. 2 and 5. What we change: DoPr preconditions the policy, while we apply it to the tokenizer to make its targets easier to predict. RobustTok perturbs a single-codebook VQ image tokenizer, while we derive TP from the mismatch-severity term and extend it to residual, scalar and prefix-conditioned quantizers.

Our main contribution is the analysis in Sec. 4, which identifies what determines downstream performance in learned action tokenization. Reconstruction only needs to be accurate enough for the task (Sec. 4.1). Chunk size and execution horizon are limited by the system, and at a fixed bitrate tokenization adds a fidelity cost (Sec. 4.2). Beyond these, the one-step error analysis (Sec. 4.3, App. C.5) shows that policy training mainly governs how often token mismatches occur, while neither the reconstruction loss nor the NLL controls how severe a mismatch is after decoding. The tokenizer also sets a floor H(C∣O)H(C \mid O) on the policy loss, which limits how predictable its tokens can be (Sec. 5.1, Eq. 3).

Sec. 5 then addresses the two issues the analysis identifies: TBR targets predictability and TP targets mismatch severity. Other methods could target the same two properties; TBR and TP are natural choices based on our analysis to adapt for action tokenizers.

4. "TBR mechanism is only shown for a linear encoder"

We agree. The formal analysis uses a linear model, and its purpose is to explain why TBR is a reasonable choice. The irreducible-loss and margin bounds in App. D.1.1 hold unchanged for nonlinear tokenizers. Linearity enters in Steps 4–6 (App. D.1.2, Assumptions A1–A3). These steps show that encoder weight along noise directions shrinks the token margins, that reconstruction penalizes this weight only weakly, and that activation preconditioning removes it faster than gradient descent. For nonlinear tokenizers, our evidence is empirical: averaged over tokenizers, TBR lowers the normalized teacher-forced NLL by 12.4%, 10.0% and 11.3% on Can, Lift and Square (ProAct vs. w/o TBR; Sec. 6.3, Fig. 3). We will also measure token margins directly on the existing tokenizers.

5. "The overall gain of using either TBR or TP on robomimic is rather low."

Each component does improve over R.O. In the overall row of Table 3 (success at Hexec=4/8), R.O. reaches 65.4/61.0, TP alone 72.0/67.9 and TBR alone 73.8/69.2, so each component alone adds 6.6 to 8.4 points, and ProAct reaches 74.5/72.3. On Can and Lift, the room for improvement is limited by strong baselines: at Hexec=4, five of the 20 R.O. configurations exceed 90% on Can and five on Lift (Tables 23–24). On Square, where the baseline is lowest (41.1/35.7), ProAct improves by 10.5/12.8 points, compared with 7.4/7.3 for TP alone and 1.2/1.3 for TBR alone (Table 3).

6. "4.1 is not a new discovery and well known in the field of robot learning. It would be helpful to explain why this is new in the context of action tokenization and autoregressive policies …" / "Tokenizer reconstruction error is not reliable to tell if the policy is successful (line 61). But is this not clear? It is the same as looking at the validation loss …"

We agree that open-loop metrics are known to be imperfect proxies for closed-loop success, including validation loss in imitation learning. Reconstruction plays a different role: it is the main training objective of learned action tokenizers (e.g., VQ-BeT, 2024; QueST, 2024; VQ-VLA, 2025; OAT, 2026). Within RVQ, NRVQ and QinCo, it is not just a noisy proxy but a misleading indicator: lower reconstruction error comes with lower success. Sec. 4.1 makes this explicit for action tokenizers, showing that their main training objective does not reflect downstream success.

What is specific to action tokenization is where this failure comes from. When no policy generates tokens, reconstruction does rank tokenizers. To measure the effect of reconstruction error directly, we pass flow-matching action chunks through each frozen tokenizer before execution, so that no token prediction is involved: reconstruction error and success then correlate strongly (Spearman ρ = −0.80, −0.41 and −0.90 on Can, Lift and Square; App. C.1.3). The gap appears once the policy has to generate the tokens. Sec. 4.3 explains why (Result 1, App. C.5). Reconstruction bounds the action error only when the policy predicts the exact encoded target, and at least 96.2% of greedy predictions on held-out data do not (App. C.4.1). For every other prediction, the error depends on what the mismatched sequence decodes to, which no standard objective controls. This tells us what a tokenizer should be trained for beyond reconstruction: targets that are easy to predict, and mismatches that decode close to the expert action. ProAct targets both.

7. "Line 66-67 how do the token sequences differ from the targets and why is this not controlled through the training objective?"

The policy's generated tokens often differ from the encoded targets: on held-out validation data, at least 96.2% of greedy predictions do (App. C.4.1). The teacher-forced NLL reduces how often this happens but cannot remove it. Similar observations can correspond to different valid token sequences, so part of the NLL, H(C∣O)H(C \mid O), is irreducible (Sec. 5.1, Eq. 3), and at rollout the policy also conditions on its own prefixes.

What lines 66–67 call uncontrolled is the resulting action error. Cross-entropy penalizes every wrong token equally, however far it decodes from the expert action, and the reconstruction loss evaluates only the encoded targets (Sec. 4.3). TP is designed to address this gap.

8. "The VLA does not predict action tokens with the VLM itself … This differs from the 'native next-token interface of autoregressive VLMs' …" / "… this makes it unclear whether ProAct's gains persist for competitive VLAs."

Our VLA follows the ActionCodec design (Dong et al., 2026; App. B.3), in which an action expert reads the VLM's context and predicts the action tokens. The action expert is still an autoregressive token predictor, so the predictability and mismatch properties of the tokenizer matter in the same way. We agree that "the native next-token interface" over-describes this setup and will revise the introduction. With the same recipe and only the tokenizer changed, ProAct improves 7 of 8 VLA pairs with no hyperparameter tuning (Table 2).

9. "It is not clear if the models trained and evaluated are a multi-seed evaluation or only using one seed. …"

Each configuration in the main sweep uses a single tokenizer seed and a single policy seed. This is stated in App. B.3 and listed as a limitation in the conclusion, and we will also state it in Sec. 6.1. The gains are consistent across configurations: ProAct improves 100 of 120 cells in Table 1, 7 of 8 VLA pairs, and all 5 real-robot configurations on both tasks (Table 4).

10. "Neither FAST nor BEAST was evaluated as a baseline."

Our study focuses on learned tokenizers, and all 22 configurations in the main comparison use the same bitrate (K = 128, L = 8, i.e., 56 bits per chunk) so that they can be compared fairly. A fair comparison with Bin, FAST or BEAST would also require matching the bitrate. App. A shows that Bin is inefficient. Most of its tokens are redundant: a policy that predicts only the first few tokens, with the rest completed by lookup, keeps about three quarters of Bin's success at Hexec=4 (Table 6). ProAct (Sec. 5) does not transfer directly to analytical tokenizers, but the analysis in Sec. 4 still applies to them.

11. Remaining questions. "Showing this ablation on the other benchmarks would help to show that it actually matters …"

We are further investigating the questions raised in the review, including a VLA whose own language-model head predicts the action tokens, longer VLA training, multiple tokenizer and policy seeds, the TBR/TP ablation on another benchmark, and FAST and BEAST at matched bitrate. We will share additional findings in this thread as they become available.

We are happy to answer further questions.

Reviewer_M8vLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper works on why accurate action reconstruction does not necessarily yield successful robot policies. It proposes ProAct, combining Token Balance Regularization to improve token predictability with Token Perturbation to improve robustness to prediction errors. Experiments cover 22 tokenizer configurations, three simulation benchmarks, a VLA model, and two real-robot tasks.

Critical strengths

The controlled study provides useful evidence that reconstruction error alone is insufficient for evaluating action tokenizers. The tokenized flow-matching and constrained-decoding experiments strengthen this finding beyond simple correlations. ProAct improves average performance across diverse tokenizer architectures and evaluation settings. Component ablations and real-robot experiments support the practical value of the approach.

Critical weaknesses
  • The methodological novelty is limited: TBR uses activation preconditioning from DoPr, while TP adapts RobustTok’s perturbation scheme. The theoretical explanation relies on restrictive linear-model assumptions, and the experiments do not directly establish that increased token margins explain the gains.
  • The main comparisons use one tokenizer-training run and one policy-training run per setting, so rollout confidence intervals do not capture training variability. Moreover, average improvements do not consistently extend to the strongest baselines: on LIBERO at execution horizon 4, the best reconstruction-only result is 73.7%, versus 72.1% for ProAct. The VLA models were also not trained to convergence.
Questions for authors
  1. What technical changes beyond adapting DoPr and RobustTok are essential to ProAct, and is there direct evidence that TBR increases token margins or assignment stability?
  2. Can you provide paired baseline–ProAct results across multiple independent tokenizer and policy training seeds for representative configurations?
  3. Do the VLA improvements persist with longer training, or do they primarily reflect faster convergence?
7 comments

Thank you for the careful review and the precise questions.

1. "The methodological novelty is limited: TBR uses activation preconditioning from DoPr, while TP adapts RobustTok's perturbation scheme." / "What technical changes beyond adapting DoPr and RobustTok are essential to ProAct …"

The main difference is that ProAct explicitly targets token predictability and mismatch robustness, rather than reconstruction quality alone. The components reuse established techniques, which we will credit more clearly in Sec. 2 and 5. DoPr applies activation preconditioning to the policy to reduce error accumulation at rollout. We apply it to the tokenizer, which is trained only for reconstruction, to change the targets the policy must predict (App. D.1). RobustTok perturbs a single-codebook VQ image tokenizer. We derive TP from the mismatch-severity term (Sec. 4.3), extend it to residual, scalar and prefix-conditioned quantizers (App. D.3), and validate it on policy-generated mismatches (Fig. 4). The two components are complementary (Table 3), and neither needs observations or the downstream policy.

Our main contribution is the analysis in Sec. 4, which identifies these two factors as important for tokenized policy learning. As you note, the controlled study shows that reconstruction error alone is insufficient, and the tokenized flow-matching and constrained-decoding experiments support this beyond correlations. Sec. 4 also shows that chunk size and execution horizon are limited by the underlying system, and that at a fixed bitrate tokenization adds a fidelity cost: success drops sharply once reconstruction error exceeds the task's tolerance (Sec. 4.2). More bitrate restores fidelity but does not reliably improve success, because it also changes the prediction problem for the policy. Beyond these, policies almost always generate sequences that differ from the encoded targets, and no standard objective controls how harmful these mismatches are after decoding (Sec. 4.3). These findings help guide the choice of bitrate, chunk size, and execution horizon. ProAct focuses on reducing the frequency and impact of token mismatches.

2. "The theoretical explanation relies on restrictive linear-model assumptions, and the experiments do not directly establish that increased token margins explain the gains." / "… is there direct evidence that TBR increases token margins or assignment stability?"

We agree that the linear analysis is a motivating mechanism, not a guarantee for deep-neural network tokenizers, and we will state this scope in Sec. 5.1. The irreducible-loss and margin bounds (App. D.1.1) hold unchanged for nonlinear tokenizers. Linearity enters only in Steps 4–6 (App. D.1.2), which compare activation preconditioning with gradient descent.

To answer your question directly: the submission does not measure margins. It shows the downstream consequence that the analysis predicts. Averaged over tokenizers, TBR lowers the normalized teacher-forced NLL by 12.4%, 10.0% and 11.3% on Can, Lift and Square, with the largest reductions at later token positions (ProAct vs. w/o TBR; Sec. 6.3, Fig. 3). We will share the results of measuring token margins later in the thread.

3. "The main comparisons use one tokenizer-training run and one policy-training run per setting …" / "Can you provide paired baseline–ProAct results across multiple independent tokenizer and policy training seeds for representative configurations?"

Correct. Each configuration in the main sweep uses one tokenizer seed and one policy seed (App. B.3), and we will state this in Sec. 6.1. The submission does include a policy-seed study, with 10 seeds for each of two tokenizers (App. B.3). The standard deviation is 1.2–2.5 points on Can and Lift and 4.3–5.8 points on Square, below ProAct's average Robomimic gain of 9.2/11.3 points. The gains are also consistent across configurations: ProAct improves 100 of 120 cells in Table 1, 7 of 8 VLA pairs and all 10 real-robot pairs. While these results show consistent improvements across configurations, we acknowledge that additional training seeds would provide a more thorough assessment of the gains.

4. "… average improvements do not consistently extend to the strongest baselines: on LIBERO at execution horizon 4, the best reconstruction-only result is 73.7%, versus 72.1% for ProAct."

You are right that ProAct does not improve the single best configuration on LIBERO at Hexec=4. All ProAct results use one hyperparameter setting per benchmark, shared across all tokenizers, because we evaluate it as a drop-in objective rather than tuning it per configuration. While ProAct does not consistently improve the best-performing configuration, it substantially improves weaker configurations and reduces the performance gap across tokenizers (Table 1). The best tokenizer design is not known before policies are trained and rolled out, so this robustness matters in practice.

5. "The VLA models were also not trained to convergence." / "Do the VLA improvements persist with longer training, or do they primarily reflect faster convergence?"

We agree that the VLA was trained for a limited budget (50k steps; App. B.3). Under that budget and recipe, with only the tokenizer changed, ProAct improves 7 of 8 pairs, from 26.2% to 48.1% average success (Table 2). The current results do not establish whether the improvements persist with longer training, and we will say so in Sec. 6.2.

6. Remaining questions. We are further investigating the questions raised in the review, including token margins and hyperparameter sensitivity, and will share additional findings in this thread as they become available.

We are happy to answer further questions.

Thank you for the clarifications. The distinction between applying preconditioning to the tokenizer and to the policy is helpful. Questions about token margins, training variability, and VLA performance after convergence remain open, but the empirical study remains valuable. I will maintain my rating of weak acceptance. Please make the relationship to prior methods and the acknowledged limitations explicit in the revised manuscript.

Reviewer_THP3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates what makes action tokenization effective for autoregressive robot policies. The authors argue that reconstruction accuracy alone is insufficient to predict downstream manipulation performance. Instead, effective tokenizers should also produce tokens that are easy for policies to predict and remain robust when predicted token sequences differ from the demonstrated ones. The authors evaluate 22 tokenizer configurations across Robomimic, LIBERO, and RoboTwin, as well as vision-language-action policies and real-world robotic manipulation. The reported results show substantial improvements over reconstruction-only training.

Critical strengths
  1. Important and well-motivated research problem. The paper challenges the common assumption that better action reconstruction necessarily leads to better downstream control. The distinction between reconstruction fidelity, token predictability, and robustness is insightful and relevant to the development of autoregressive robot policies.

  2. Comprehensive empirical investigation. The evaluation covers 22 tokenizer configurations involving different encoder architectures and quantization methods. This systematic comparison provides useful evidence about the relationship between tokenizer design and downstream policy performance.

  3. Strong experimental improvements. ProAct improves average simulation success by 11.3 percentage points at an execution horizon of eight steps. The results also transfer to VLA policies, improving average LIBERO success from 26.2% to 48.1%. The real-robot experiments further demonstrate substantial gains across the evaluated configurations.

Critical weaknesses
  1. The technical novelty is somewhat limited. Although the overall perspective is interesting, the two proposed components build on relatively established ideas. TBR relies on covariance-aware feature preconditioning, while TP resembles noise injection or denoising-based robustness training. The authors should more clearly distinguish their contributions from existing representation regularization and robust quantization methods.

  2. The theoretical analysis relies on restrictive assumptions. The relationship between balanced feature learning, token margins, and predictability is primarily established under a linear encoder assumption. It is unclear how directly these conclusions extend to the nonlinear transformer and convolutional tokenizers used in the experiments.

  3. The causal explanation for TBR could be stronger. The authors report that TBR reduces prediction NLL, but the paper does not directly establish that larger token margins are responsible for this improvement. Additional measurements of token margins, feature covariance, and conditional token entropy would better connect the theoretical motivation with the empirical results.

  4. Some experimental improvements are inconsistent. Although ProAct improves average performance, Table 1 shows several configurations where it reduces downstream success, including C2-RVQ-S on RoboTwin. The VLA experiments also show a decline for TF-RVQ-S. The paper would benefit from a more detailed analysis of when and why ProAct fails to improve performance.

  5. The real-world evaluation remains limited. The real-robot experiments involve two manipulation tasks on a UR5 robot. Although the results are promising, broader evaluations across different robot embodiments, more challenging manipulation tasks, and environmental disturbances would strengthen the generalization claims.

Questions for authors

please refer to the weakness part.

6 comments

Thank you for the positive assessment and the concrete suggestions.

1. "… TBR relies on covariance-aware feature preconditioning, while TP resembles noise injection or denoising-based robustness training. The authors should more clearly distinguish their contributions from existing representation regularization and robust quantization methods."

The main difference is that ProAct explicitly targets token predictability and mismatch robustness, rather than reconstruction quality alone (Sec. 4.3). The components reuse established techniques, which we will credit more clearly in Secs. 2 and 5. DoPr applies activation preconditioning when training the policy, to reduce error accumulation at test time; we apply it to the tokenizer, which is trained only for reconstruction, to change the targets the policy must predict (App. D.1–D.2). RobustTok perturbs a single-codebook VQ image tokenizer; we extend its sampler to residual, normalized, scalar and prefix-conditioned quantizers (App. D.3) and validate it on policy-generated mismatches (Fig. 4).

Compared with representation regularization such as batch normalization or whitening (Sec. 5.1), TBR changes only the optimization: the tokenizer's architecture, forward computation and loss stay the same. Existing robust quantization methods, such as StableToken (Song et al., 2026) for speech, aim to keep token assignments stable under input noise. Our focus is different: we aim to make tokens easier for the policy to predict and reduce the decoding errors caused by incorrect predictions. In particular, TP replaces tokens with nearby quantizer candidates during training rather than adding continuous noise to the input (App. D.3).

2. "The theoretical analysis relies on restrictive assumptions. … It is unclear how directly these conclusions extend to the nonlinear transformer and convolutional tokenizers used in the experiments."

We agree. The linear analysis is a motivating mechanism, not a guarantee for deep tokenizers, and we will state this scope in Sec. 5.1. The irreducible-loss and margin bounds in App. D.1.1 hold unchanged for nonlinear tokenizers. Linearity enters in Steps 4–6 (App. D.1.2, Assumptions A1–A3), which show that encoder weight along noise directions shrinks the token margins, that reconstruction penalizes this weight only weakly, and that activation preconditioning removes it faster than gradient descent. For the nonlinear tokenizers, the evidence is empirical: averaged over tokenizers, TBR lowers the normalized teacher-forced NLL by 12.4%, 10.0% and 11.3% on Can, Lift and Square (ProAct vs. w/o TBR; Sec. 6.3, Fig. 3).

3. "… the paper does not directly establish that larger token margins are responsible for this improvement. Additional measurements of token margins, feature covariance, and conditional token entropy would better connect the theoretical motivation with the empirical results."

Thank you for this suggestion. TBR lowers the teacher-forced NLL and makes prediction difficulty more uniform across positions: the spread of NLL across positions falls by 18.9–21.4% (Sec. 6.3, Fig. 3). Since the policy loss upper-bounds the conditional token entropy (Sec. 5.1, Eq. 3), this is consistent with lower conditional entropy. We agree, however, that it does not show that larger margins are the cause. We will measure the quantities you list (token margins, feature-covariance spectra and conditional token entropy) on the existing tokenizers with and without TBR, and relate them to the per-position NLL changes in Fig. 3. We are currently analyzing token margins, feature covariance, and conditional token entropy using the existing tokenizers, and we will share the results once they are ready.

4. "… Table 1 shows several configurations where it reduces downstream success, including C2-RVQ-S on RoboTwin. The VLA experiments also show a decline for TF-RVQ-S. The paper would benefit from a more detailed analysis of when and why ProAct fails to improve performance."

We agree, and we will add a failure-case analysis paragraph. All ProAct results use one hyperparameter setting per benchmark, shared across tokenizers. Under this setting, ProAct is worse than R.O. in 18 of the 120 cells of Table 1, and three patterns account for most of them.

First, strong baselines gain less. 15 of the 18 cells are shared-codebook configurations, which mostly have strong R.O. baselines. C2-RVQ-S is the best R.O. tokenizer on Robomimic (85.1% at Hexec=4) and the second best on LIBERO (70.9%). ProAct roughly preserves its performance there (−3.1/+0.6 and −0.8/−1.3 points), leaving little room for improvement. TF-RVQ-S is likewise the strongest R.O. tokenizer in the VLA experiment (53.8%).

Second, TBR is weaker for convolutional tokenizers. 11 of the 18 cells are Conv2D configurations. For convolutional layers, TBR uses a rank-one-plus-diagonal approximation of the activation covariance instead of the full covariance (App. D.2, Eq. 60), so it balances features less than in the other backbones under the same hyperparameter setting.

Third, RoboTwin may be more sensitive to hyperparameters. 10 of the 18 cells are on RoboTwin, where demonstrations are generated by scripted experts and show relatively low action variability (translation std 0.00507, rotation std 0.02957, as in submission line 1287). We suspect this may make the choice of perturbation strength more sensitive, but have not yet established a direct link to the performance drops. We are further investigating this and will share additional findings if available.

5. "The real-world evaluation remains limited. The real-robot experiments involve two manipulation tasks on a UR5 robot. …"

We agree that two tasks on one embodiment cannot establish broad generalization, and we will state this in the limitations. The primary motivation for our real-robot experiments is to verify whether the improvements observed in simulation carry over to real-world settings, where there is noise and latency. Across five tokenizer configurations and two tasks, ProAct improves all 10 tokenizer–task pairs, raising average success from 18.7% to 55.3% (Table 4), with the largest gains for level-specific variants on Cube Stacking, as in simulation. Broader embodiments and tasks types are important future work.

6. Remaining questions. We are further investigating the questions raised in the review, including token margins, feature covariance, and hyperparameter sensitivity for configurations where ProAct underperforms on RoboTwin. We will update this thread as we get answers to these questions.

We are happy to answer further questions.

Following up on 3. "… the paper does not directly establish that larger token margins are responsible for this improvement. Additional measurements of token margins, feature covariance, and conditional token entropy would better connect the theoretical motivation with the empirical results."

We measured all three quantities on the 22 tokenizer configurations, each trained with and without TBR (ProAct vs. w/o TBR, Robomimic), using 512 held-out action chunks per task.

Token margins. In App. D.1.1, the margin bounds the label inconsistency p≠p_{\ne}, which controls the irreducible loss, so we estimate p≠p_{\ne} directly. For a held-out normalized chunk aa, we add Gaussian noise ε\varepsilon rescaled to ∥ε∥2=1\lVert \varepsilon \rVert_2 = 1, with 16 draws per chunk shared by both tokenizers. For each position ii we measure

qi=Pr⁡[ci(a+ε)≠ci(a)∣c<i(a+ε)=c<i(a)],q_i = \Pr\left[c_i(a + \varepsilon) \ne c_i(a) \mid c_{\lt i}(a + \varepsilon) = c_{\lt i}(a)\right],

the probability that token ii changes given that all earlier tokens are unchanged.

  • With the recorded chunk as the reference action g(O)g(O) (the bounds hold for any gg), qiq_i is p≠p_{\ne} at position ii.
  • Token ii can change only if its margin mi(a)m_i(a) is at most ∥ε∥2\lVert \varepsilon \rVert_2, so a lower qiq_i means larger margins.

Table R1. qiq_i (%), averaged over the 22 configurations and 3 tasks. The last row is the relative change in teacher-forced NLL from Fig. 3. Negative Δ\Delta favors ProAct.

Position12345678
w/o TBR8.216.025.933.139.142.847.949.2
ProAct7.317.225.531.235.838.441.542.2
Δqi\Delta q_i (pts)−0.9+1.2−0.4−1.9−3.3−4.4−6.4−7.0
Δ\Delta NLL (%)−1.8+6.8+1.2−8.0−11.7−14.8−15.8−16.5

TBR lowers qiq_i at every position except position 2, by up to 7.0 points at position 8.

Conditional token entropy. The teacher-forced NLL at each position upper-bounds the conditional token entropy there (Sec. 5.1, Eq. 3). The NLL drops most where qiq_i drops most, from 8% at position 4 to 17% at position 8. The change in qiq_i correlates with the change in NLL (Spearman ρ\rho):

  • ρ=0.75\rho = 0.75 across all 528 combinations of configuration, task and position, each pairing Δqi\Delta q_i with the NLL change at the same position;
  • ρ=0.75\rho = 0.75 when comparing configurations at the same position and task, so the correlation is not only because both drop more at later positions;
  • ρ=0.81\rho = 0.81 across the 22 configurations.

The link also holds across quantizer families, and it extends to success (Table R2). Where every pair becomes more stable (QinCo, VQ, FSQ), TBR also lowers the NLL by 23–29% and gives the largest success gains. For NRVQ and RVQ, stability, NLL and success change little.

Table R2. Effect of TBR by quantizer family (ProAct vs. w/o TBR, Robomimic). "More stable" means TBR lowers the probability that any of the 8 tokens changes, under the same noise as Table R1.

QuantizerPairs more stableΔ\Delta NLL (%)Δ\Delta success, execution horizon 4 / 8 (pts)
QinCo18 / 18−29.3+7.3 / +9.0
VQ6 / 6−28.8+2.8 / +4.8
FSQ6 / 6−22.9+2.9 / +6.1
NRVQ8 / 18−0.6−1.5 / +0.5
RVQ3 / 18+3.0+0.3 / +1.3
All41 / 66−11.1+2.2 / +3.9

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.