- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes ActionCodec2, a streaming action tokenizer for autoregressive VLAs. Unlike existing chunk-based tokenizers, ActionCodec2 allows actions to be decoded and executed as tokens are generated. The method uses error-feedback quantization to keep trajectory reconstruction errors within a given tolerance, and formulates the minimum required action vocabulary as a geometric covering problem. It also introduces Set-BPE to compress action tokens across dimensions and time steps. Extensive experiments show strong performance across simulation benchmarks and real-world robotic tasks. The authors further demonstrate how action tokens can serve as a control interface for VLM agents, allowing them to build and reuse motion macros.
Critical strengths
-
The paper is well organized and easy to follow. The figures and tables are also clear and well designed.
-
I find the motivation behind streaming tokenization and bounded tracking error interesting. The geometric covering formulation makes sense, and the theoretical results directly guide the method design. I appreciate this approach, as it is more convincing than simply stacking additional neural network modules.
-
The experiments are solid, covering multiple simulation benchmarks and real-world robot experiments. The results demonstrate the advantages of ActionCodec2 in imitation learning, reinforcement learning, and streaming execution.
-
I find the Agent Interface experiments particularly interesting. The authors show that frontier LLMs can compose action tokens into reusable macros and gradually improve them across attempts. This is a novel and interesting way to use action tokens beyond conventional VLA training, and I think it opens up some promising possibilities for future research.
Critical weaknesses
-
Some parts of the paper are difficult to understand, especially the theoretical sections. There are many definitions and proofs, but not enough intuitive explanations or examples. I think the authors could make the theory much easier to follow with a few simple examples.
-
The authors claim that streaming tokenization enables closed-loop feedback within an action chunk, allowing the policy to adjust its actions during execution. This makes sense, but the paper lacks experiments that directly demonstrate this capability. I would like to see how ActionCodec2 performs when the environment changes or unexpected disturbances occur during chunk execution, and whether streaming feedback actually improves the policy's ability to react to these changes.
-
Although the authors provide a pretrained universal version of ActionCodec2, its vocabulary size and tracking-error tolerance are fixed. Different robots and tasks may require very different levels of precision and control frequencies. I am not sure whether a single fixed configuration can meet all these requirements in practice.
Questions for authors
-
Could the authors provide a simple worked example illustrating how the covering formulation determines the primitive set and prevents trajectory drift? Please also clarify the scope of the optimality guarantee, particularly the role of the full tolerance requirement.
-
Does the evaluated streaming policy incorporate new observations and revise subsequent actions during execution, or does it only release actions earlier from an otherwise fixed prediction? What experimental evidence demonstrates improved reactions to disturbances introduced during chunk execution, beyond the latency benefits shown in Section 7.6?
-
How should the pretrained universal tokenizer be adapted to robots or tasks requiring different tracking tolerances and control frequencies? Would this require rebuilding the primitive set, refitting Set-BPE, or retraining the VLA? Please clarify the trade-offs among precision, token rate, and compatibility with the pretrained vocabulary.