acceptodds
Withdrawn from ICLR 2027

Residual Action Tokenization for Autoregressive Robot Policies

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autoregressive robot policies rely on action tokenizers to convert continuous action chunks into compact sequences of discrete prediction targets. Existing tokenizers can reconstruct action chunks from discrete codes, but often leave implicit what each individual token contributes to the reconstructed action. We introduce Residual Action Tokenization (ReAT), which gives each token an explicit role in refining the current action reconstruction. It uses the decoded reconstruction as feedback to construct the next token from the remaining action error. This closes the loop between encoding and decoding, making each new token responsive to the errors left by the actual decoded prefix. Across 19 manipulation tasks spanning three simulation benchmarks and real-world settings, 8-token ReAT policies outperform the evaluated baselines in task success; on LIBERO, even 4-token ReAT surpasses the evaluated 8-token action-tokenization baselines at half the token budget. These results suggest a path toward action tokenization that supports self-corrective policy learning, helping autoregressive policies learn to progressively refine their own action plans.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, robot manipulation, action tokenization, autoregressive policies, residual quantization, imitation learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CoXHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReAT, an action tokenization network based on residual quantization. Existing methods follow an tokens-in actions-out paradigm. The proposed method allows for multiple quantization passes and each pass encode the residual quantization error from previous rounds. The proposed techniques are evaluated on LIBERO, MetaWorld and RoboCasa and shows advantages over existing strategy such as FAST, OAT.

Critical strengths

The concept of having a close-loop action tokenizer that is aware of quantization error is a very interesting one. I like the novelty of the proposed approach.

Critical weaknesses

I feel like the paper is not exactly well written. Too many small paragraphs with lots of empty spaces left. It almost looks like the author has not much things to say and do space padding. Explanation also not exactly clear, like the state, update in algorithm 1 block and LprogressL_{progress}, LregL_{reg} are not properly explained.

My second concern is the solidness of evaluation. Table 2 shows the evaluation results on LIBERO, MetaWorld, RoboCasa, yet the overall numbers are very low. Like LIBERO best performance is only 71% and Metaworld is only 31%. Yet both of these two benchmarks are well regarded as saturated by today's standard. This makes me skeptical about the results.

Questions for authors

See Weaknesses.

Reviewer_GRq4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a new architecture for Autoregressive Robot Policies - it autoregressively predicts residual-refinement tokens rather than successive action steps. It has shown promising results in comparison with autoregressive baselines, both in simulation and real-world experiments. The proposed method does not only show benefit in success rate, but also achieves a strong success-rate/latency tradeoff.

Critical strengths
  • The proposed method models the autoregressive action prediction problem in a well-motivated and novel manner - intead of directly predicting action tokens, ReAT predicts residual-refinement tokens, to progressively correct the decoded action trunks.
  • ReAT has shown promising results in comparison with autoregressive baselines, both in simulation and real-world experiments.
  • ReAT does not only show strong benefit in success rate, but also achieves a strong success-rate/latency tradeoff. The real-world bucket stacking experiment shows very strong improvement from ReAT.
Critical weaknesses
  • The claim that "gives each token an explicit role in refining the current action reconstruction" can be misleading. It reads like each token as a separate, semantically interpretable role. Suggest to soften that wording to reduce confusion and make their collective corrective role more clear.
  • Although ReAT performs strongly on real-world bucket stacking experiment, the overall physical evaluation is quite narrow. Would be great to extend to other physical experiments.
Questions for authors
  • Can you provide some case study (preferrably in real-world setting) to give some insights about how each of those tokens progressives corrects the predicted action trunks?
  • Can you provide some analysis on in-distribution vs out-of-distribution performance of ReAT?
Reviewer_oKPsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors introduce ReAT, an action tokenization method for autoregressive robot policies in which each new token encodes the residual error of the current action reconstruction. The method combines a draft-conditioned recurrent decoder with prefix reconstruction supervision to turn residual tokens into successive action corrections, allowing a single policy to trade off action quality and inference latency by varying the token budget without retraining. Across 19 manipulation tasks on three simulation benchmarks and a real-world bucket-stacking task, ReAT outperforms the evaluated baselines at eight tokens.

Critical strengths

[Critical Strengths]

  1. The authors outperform previous methods across extensive datasets.

  2. The authors introduce ReTA, which leverages a learned tokenizer that explicitly constructs each token as a corrective residual.

[Minor Strengths]

  1. The authors conduct extensive experiments to demonstrate their contributions.

  2. The authors attached a clear algorithm summary, which makes it easy to understand.

Critical weaknesses

[Critical Weakness]

  1. The authors claim that the recurrent decoder balances inference latency and action quality but provide insufficient evidence. Please report FLOPs, decoding speed, and latency under different token budgets.

  2. The proposed approach appears to adapt established autoregressive techniques from NLP with limited methodological innovation. The authors should clarify its novelty and compare it with relevant methods (e.g., autoregressive LLMs, speculative decoding, Autoregressive Diffusion Model). Even if they do not attach it, the authors should mention several related papers.

  3. The paper lacks hyperparameter analysis. Evaluate the sensitivity to key parameters, such as the locality edit weight.

  4. Figure 3 does not clearly illustrate how the proposed method balances action quality and inference latency. Please revise the figure for clarity.

  5. The experiments focus on static objects. Please evaluate ReAT in dynamic and occluded scenarios to demonstrate its robustness and generalizability.

  6. The paper lacks token-level statistical analysis, such as codebook utilization, token entropy, and n-gram predictability, which could help characterize the learned representations.

  7. The latency analysis focuses on deployment costs but lacks test-time latency measurements across the evaluated datasets. Please report inference latency on all benchmarks to improve reproducibility.

  8. Evaluating Diffusion Policy with only 10 DDIM sampling steps may underestimate its potential performance. Please evaluate different sampling-step configurations and compare their performance–latency trade-offs.

  9. The authors should evaluate the proposed approach with diverse diffusion policy models to demonstrate its generalizability across different baselines.

[Minor Weakness]

  1. Five random seeds may be insufficient to show the generalizability. The authors should conduct additional experiments with 10–20 seeds to better assess the variability and robustness of the results.

  2. The ablation study reports numerical results (e.g., lines 337 and 407), but the contributions of individual components are not clearly explained. The authors should provide more detailed analysis of how each component contributes to the overall performance.

  3. In Figure 5 and Table 4, the alignment and placement of figures and tables could be improved. The current layout appears inconsistent with the paper template.

Questions for authors
  1. How many layers are used, and how is the network designed for the Encoder and Decoder?

Comments on the paper

1 comment
~Vaibhav_Rathore1OpenReview ↗

Evaluation Protocol Configurations ?

Thank you for this interesting work. We work on action tokenization and would like to compare results across papers carefully. Could the authors clarify a few evaluation details?

  1. LIBERO: is the first policy observation taken before or after the environment's settling steps?
  2. "Mean ± SEM across five evaluation rounds": are these five rollout rounds of one trained policy, or five training seeds? How many episodes per round?
  3. Decoding: do the OAT and ReAT rows use greedy decoding or top-k sampling (with which k and temperature)?
  4. MetaWorld: which MetaWorld version, demonstration source, camera and episode limit were used, and are velocities cleared at reset? Per-task results would be very helpful, since the reported OAT8 (29.9) differs from the OAT paper (24.4).
  5. RoboCasa: which tasks and which dataset release?
  6. OAT baselines: how were checkpoints selected, and are OAT2 and OAT4 truncated decodings of the OAT8 policy or separately trained models? (The reported OAT2 on LIBERO, 33.8, differs from the OAT paper's 39.8.)
  7. Which quantizer does each ReAT token use, and with what codebook size?
  8. Do you plan to release code and evaluation scripts?

Thank you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.