- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies learned action tokenizer for autoregressive robot policies and evaluates over 22 different tokenizer configurations. The benchmarks used for evaluation are Robomimic, LIBERO, RoboTwin and real robot experiments, plus a VLA experiment on LIBERO. The study reveals that, beyond a sufficient level of accuracy, reconstruction error for action chunks does not predict how successful a policy is during rollouts and it separates matched token predictions from mismatches. The paper introduces an improved method for action tokenizer called ProAct. ProAct uses token balance regularization and token perturbation to improve results of most evaluated tokenizers on the three different benchmarks without needing environment observations or other robot related data beside the action chunks.
Critical strengths
- Broad evaluation across many different benchmarks, configurations and models
- Easy to follow and well motivated paper
Critical weaknesses
- The framing doesn't match the evaluation setup. The work is motivated through autoregressive VLMs and VLAs, yet all evaluations use a small 4-layer, d=256 transformer trained from scratch, which is closer to BeT, VQ-BeT and QueST than to a VLA. The main architecture is not disclosed in the main text.
- The VLA does not predict action tokens with the VLM itself: a separate, randomly initialized 43M Llama action expert predicts the 8 tokens from a 128-token action vocabulary, using only the VLM’s keys and values (Appendix Table 12). This differs from the ‘native next-token interface of autoregressive VLMs’ used to motivate action tokenization in the introduction.
- Both ProAct components mainly build on existing methods. TP adapts RobustTok and TBR is an existing preconditioner. Clarifications regarding novelty beyond motivation and application would strengthen the contribution.
- TBR mechanism is only shown for a linear encoder
- The reported 48.1% on LIBERO is far below published results. We acknowledge the smaller model and limited training budget, but this makes it unclear whether ProAct’s gains persist for competitive VLAs.
- It is not clear if the models trained and evaluated are a multi-seed evaluation or only using one seed. If only one seed was used the claims are weak as it is not known if other models are within variance/noise -The overall gain of using either TBR or TP on robomimic is rather low. Showing this ablation on the other benchmarks would help to show that it actually matters or analyzing more in what context ProAct actually improves action tokenization.
- 4.1 is not a new discovery and well known in the field of robot learning. It would be helpful to explain why this is new in the context of action tokenization and autoregressive policies to make it more clear.
- Neither FAST nor BEAST was evaluated as a baseline.
- Paper is generally well written, but key details are only in the appendix (Policy details, VLA details)
Questions for authors
- Tokenizer reconstruction error is not reliable to tell if the policy is successful (line 61). But is this not clear? It is the same as looking at the validation loss and saying the policy is good because the loss is low. This is all in open loop and does not say anything about performance.
- Line 66-67 how do the token sequences differ from the targets and why is this not controlled through the training objective?
- Could you explain what exactly the novelty of the work is and what parts already exist in their form and which are newly proposed?
9 comments
6. "4.1 is not a new discovery and well known in the field of robot learning. It would be helpful to explain why this is new in the context of action tokenization and autoregressive policies …" / "Tokenizer reconstruction error is not reliable to tell if the policy is successful (line 61). But is this not clear? It is the same as looking at the validation loss …"
We agree that open-loop metrics are known to be imperfect proxies for closed-loop success, including validation loss in imitation learning. Reconstruction plays a different role: it is the main training objective of learned action tokenizers (e.g., VQ-BeT, 2024; QueST, 2024; VQ-VLA, 2025; OAT, 2026). Within RVQ, NRVQ and QinCo, it is not just a noisy proxy but a misleading indicator: lower reconstruction error comes with lower success. Sec. 4.1 makes this explicit for action tokenizers, showing that their main training objective does not reflect downstream success.
What is specific to action tokenization is where this failure comes from. When no policy generates tokens, reconstruction does rank tokenizers. To measure the effect of reconstruction error directly, we pass flow-matching action chunks through each frozen tokenizer before execution, so that no token prediction is involved: reconstruction error and success then correlate strongly (Spearman ρ = −0.80, −0.41 and −0.90 on Can, Lift and Square; App. C.1.3). The gap appears once the policy has to generate the tokens. Sec. 4.3 explains why (Result 1, App. C.5). Reconstruction bounds the action error only when the policy predicts the exact encoded target, and at least 96.2% of greedy predictions on held-out data do not (App. C.4.1). For every other prediction, the error depends on what the mismatched sequence decodes to, which no standard objective controls. This tells us what a tokenizer should be trained for beyond reconstruction: targets that are easy to predict, and mismatches that decode close to the expert action. ProAct targets both.
7. "Line 66-67 how do the token sequences differ from the targets and why is this not controlled through the training objective?"
The policy's generated tokens often differ from the encoded targets: on held-out validation data, at least 96.2% of greedy predictions do (App. C.4.1). The teacher-forced NLL reduces how often this happens but cannot remove it. Similar observations can correspond to different valid token sequences, so part of the NLL, , is irreducible (Sec. 5.1, Eq. 3), and at rollout the policy also conditions on its own prefixes.
What lines 66–67 call uncontrolled is the resulting action error. Cross-entropy penalizes every wrong token equally, however far it decodes from the expert action, and the reconstruction loss evaluates only the encoded targets (Sec. 4.3). TP is designed to address this gap.
8. "The VLA does not predict action tokens with the VLM itself … This differs from the 'native next-token interface of autoregressive VLMs' …" / "… this makes it unclear whether ProAct's gains persist for competitive VLAs."
Our VLA follows the ActionCodec design (Dong et al., 2026; App. B.3), in which an action expert reads the VLM's context and predicts the action tokens. The action expert is still an autoregressive token predictor, so the predictability and mismatch properties of the tokenizer matter in the same way. We agree that "the native next-token interface" over-describes this setup and will revise the introduction. With the same recipe and only the tokenizer changed, ProAct improves 7 of 8 VLA pairs with no hyperparameter tuning (Table 2).
9. "It is not clear if the models trained and evaluated are a multi-seed evaluation or only using one seed. …"
Each configuration in the main sweep uses a single tokenizer seed and a single policy seed. This is stated in App. B.3 and listed as a limitation in the conclusion, and we will also state it in Sec. 6.1. The gains are consistent across configurations: ProAct improves 100 of 120 cells in Table 1, 7 of 8 VLA pairs, and all 5 real-robot configurations on both tasks (Table 4).
10. "Neither FAST nor BEAST was evaluated as a baseline."
Our study focuses on learned tokenizers, and all 22 configurations in the main comparison use the same bitrate (K = 128, L = 8, i.e., 56 bits per chunk) so that they can be compared fairly. A fair comparison with Bin, FAST or BEAST would also require matching the bitrate. App. A shows that Bin is inefficient. Most of its tokens are redundant: a policy that predicts only the first few tokens, with the rest completed by lookup, keeps about three quarters of Bin's success at Hexec=4 (Table 6). ProAct (Sec. 5) does not transfer directly to analytical tokenizers, but the analysis in Sec. 4 still applies to them.
11. Remaining questions. "Showing this ablation on the other benchmarks would help to show that it actually matters …"
We are further investigating the questions raised in the review, including a VLA whose own language-model head predicts the action tokens, longer VLA training, multiple tokenizer and policy seeds, the TBR/TP ablation on another benchmark, and FAST and BEAST at matched bitrate. We will share additional findings in this thread as they become available.
We are happy to answer further questions.
Thank you for the detailed review.
1. "The framing doesn't match the evaluation setup. … yet all evaluations use a small 4-layer, d=256 transformer trained from scratch …"
The paper is about autoregressive policies, including VLAs. The main sweep uses a compact autoregressive policy (Table 10) because it is a controlled study: the policy recipe is fixed and only the tokenizer changes. A study of 22 configurations across 3 benchmarks with VLAs is beyond our compute budget. Not all of our evaluations use the compact policy, however. In the VLA experiment, ProAct improves 7 of 8 tokenizer pairs, from 26.2% to 48.1% average success (Sec. 6.2, Table 2), which suggests that the findings of the controlled study carry over.
2. "The main architecture is not disclosed in the main text." / "Paper is generally well written, but key details are only in the appendix …"
Thank you for finding the paper well written. The revision allows one additional page, so we will move the main policy and VLA architecture and training details into Sec. 6.1 and keep the full tables in App. B.3 (Tables 10 and 12).
3. "Both ProAct components mainly build on existing methods. TP adapts RobustTok and TBR is an existing preconditioner. …" / "… what parts already exist in their form and which are newly proposed?"
We agree, and we will separate the two more clearly in the paper. What already exists: TBR uses the activation-preconditioning component of DoPr (App. D.2), and the TP sampler is adapted from RobustTok (App. D.3); we will credit both more clearly in Sec. 2 and 5. What we change: DoPr preconditions the policy, while we apply it to the tokenizer to make its targets easier to predict. RobustTok perturbs a single-codebook VQ image tokenizer, while we derive TP from the mismatch-severity term and extend it to residual, scalar and prefix-conditioned quantizers.
Our main contribution is the analysis in Sec. 4, which identifies what determines downstream performance in learned action tokenization. Reconstruction only needs to be accurate enough for the task (Sec. 4.1). Chunk size and execution horizon are limited by the system, and at a fixed bitrate tokenization adds a fidelity cost (Sec. 4.2). Beyond these, the one-step error analysis (Sec. 4.3, App. C.5) shows that policy training mainly governs how often token mismatches occur, while neither the reconstruction loss nor the NLL controls how severe a mismatch is after decoding. The tokenizer also sets a floor H(C∣O) on the policy loss, which limits how predictable its tokens can be (Sec. 5.1, Eq. 3).
Sec. 5 then addresses the two issues the analysis identifies: TBR targets predictability and TP targets mismatch severity. Other methods could target the same two properties; TBR and TP are natural choices based on our analysis to adapt for action tokenizers.
4. "TBR mechanism is only shown for a linear encoder"
We agree. The formal analysis uses a linear model, and its purpose is to explain why TBR is a reasonable choice. The irreducible-loss and margin bounds in App. D.1.1 hold unchanged for nonlinear tokenizers. Linearity enters in Steps 4–6 (App. D.1.2, Assumptions A1–A3). These steps show that encoder weight along noise directions shrinks the token margins, that reconstruction penalizes this weight only weakly, and that activation preconditioning removes it faster than gradient descent. For nonlinear tokenizers, our evidence is empirical: averaged over tokenizers, TBR lowers the normalized teacher-forced NLL by 12.4%, 10.0% and 11.3% on Can, Lift and Square (ProAct vs. w/o TBR; Sec. 6.3, Fig. 3). We will also measure token margins directly on the existing tokenizers.
5. "The overall gain of using either TBR or TP on robomimic is rather low."
Each component does improve over R.O. In the overall row of Table 3 (success at Hexec=4/8), R.O. reaches 65.4/61.0, TP alone 72.0/67.9 and TBR alone 73.8/69.2, so each component alone adds 6.6 to 8.4 points, and ProAct reaches 74.5/72.3. On Can and Lift, the room for improvement is limited by strong baselines: at Hexec=4, five of the 20 R.O. configurations exceed 90% on Can and five on Lift (Tables 23–24). On Square, where the baseline is lowest (41.1/35.7), ProAct improves by 10.5/12.8 points, compared with 7.4/7.3 for TP alone and 1.2/1.3 for TBR alone (Table 3).