- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes HoloBrain-Discrete, which builds on HoloBrain’s joint-centric action representation and graph attention, introducing a shared virtual-joint tokenizer and a delayed autoregressive policy that predicts discrete action codes, which are subsequently decoded into continuous-valued joint and link-pose trajectories. Its main contributions are a shared virtual-joint tokenizer that compresses joint and link-pose trajectories across embodiments, and a code/cue delayed autoregressive policy that predicts token groups across temporal, virtual-joint, and residual-refinement axes. Evaluation uses approximately 10,000 hours of data spanning 18 embodiments, simulation benchmarks, limited-demonstration co-training, and seven real-world tasks. The method achieves stronger performance at measured pretraining checkpoints, improved cross-embodiment co-training results, and approximately 1.9–2.1× faster model inference than HoloBrain. However, post-training gains are mixed, with lower average real-world progress.
Critical strengths
Seems useful approach to compressing structured, heterogeneous action spaces. The virtual-joint bottleneck preserves access to robot connectivity while sharing tokenizer parameters across embodiments.
Critical weaknesses
-
The comparison omits a particularly relevant competing method, FASTer. Although acknowledged in Section 2, FASTer is absent from the representation and policy evaluations. Its learned residual tokenizer and blockwise autoregressive generation address closely related problems, making it especially relevant for assessing the additional value of virtual-joint tokenization and code/cue delayed generation.
-
The stronger pretraining results do not consistently translate into better deployed policies. Post-training gains remain uneven, with lower real-world manipulation scores.
Questions for authors
Table 10 shows substantial real-world progress deficits on Piper relative to HoloBrain for towel folding (−29.4 points) and bottle disposal (−19.5 points). Is it possible to do targeted failure analysis or ablations to identify whether these deficits primarily originate in perception, token prediction, tokenizer reconstruction, or control?