- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ARPG separates action representation learning from policy generation. It pretrains an action encoder by aligning human hand trajectories with frozen video features, then freezes the encoder and trains a robot-specific decoder and a conditional flow model on robot demonstrations. Evaluation covers DexJoCo, three physical manipulation tasks and representation ablations. A nested human-data study finds improved downstream success from 10 to 500 hours, with no further gain at 3,000 hours.
Critical strengths
The representation ablation is the strongest evidence for the method: visual alignment reaches 57.8%, versus 36.4% for Action-MAE and 30.5% for Action-iBOT, with robot data, observations and the downstream generator held fixed (Table 2). This makes the benefit more convincing than the headline comparison with larger policies alone.
The gains extend to physical manipulation: ARPG improves all three real-robot tasks, including coffee-bean pouring from 20.0% to 53.3% over 30 trials per method (Fig. 3). Together with the 44.7% multitask bimanual result on DexJoCo and a 0.46B-parameter deployed model, this gives the approach practical value.
Critical weaknesses
The headline simulation comparison does not establish that execution settings are matched. Baselines are taken from published DexJoCo results, while ARPG uses strict RTC with guidance from the previous action chunk (App. F); without a matched execution comparison, the contribution of this guidance to the bimanual gain remains unclear.
The pretraining budget for the scaling curves is unclear: Table 6 lists both 21,714 steps and three epochs, which cannot both stay fixed as the dataset grows. Since Fig. 4(c) shows that performance depends on training duration, the per-scale schedules are needed to interpret the data-expansion result.
Questions for authors
Does ARPG retain its bimanual advantage with matched action-chunk execution and RTC settings, or when its previous-chunk guidance is disabled? This would separate the representation benefit from execution changes.
What data subset, update count and checkpoint were used for each point in Fig. 4 and for the main model? Please clarify whether the data sweep fixes steps or epochs, and how Table 6 relates to the best reported 10k-step point in Fig. 4(c).
4 comments
Response to Reviewer 4g9R(Part 2/3)
W2 & Q2: Clarification of Scaling Experiment Configurations and Checkpoint Selection
What data subset, update count and checkpoint were used for each point in Fig. 4 and for the main model? Please clarify whether the data sweep fixes steps or epochs, and how Table 6 relates to the best reported 10k-step point in Fig. 4(c).
We sincerely apologize for the confusion caused by our original manuscript. We acknowledge that the relationship between these experimental settings was not clearly explained: Table 7 reports the complete pretraining configuration for the action encoder trained on 3,000 hours of data, whereas the scaling experiments in Figure 5 use fixed pretraining checkpoints for downstream evaluation.
We have carefully reviewed and summarized the exact configurations for each data point and the main models as follows:
| Location / Data Point | Encoder Pretraining Data | Encoder Parameters | Encoder Checkpoint (Updates) | DiT Checkpoint (Updates) |
|---|---|---|---|---|
| Fig. 5(a): Robot-only | DexJoCo, 8.8h | 22M | 10,000 | 120,000 |
| Fig. 5(a): Human 10h | Human-only, 10h | 22M | 10,000 | 120,000 |
| Fig. 5(a): Human 100h | Human-only, 100h | 22M | 10,000 | 120,000 |
| Fig. 5(a): Human 500h | Human-only, 500h | 22M | 10,000 | 120,000 |
| Fig. 5(a): Human 3000h | Human-only, ~3,000h | 22M | 10,000 | 120,000 |
| Fig. 5(b): 22M | Human-only, 500h | 22M | 10,000 | 120,000 |
| Fig. 5(b): 40M | Human-only, 500h | 40M | 10,000 | 120,000 |
| Fig. 5(b): 90M | Human-only, 500h | 90M | 10,000 | 120,000 |
| Fig. 5(c): 10k | Human-only, 500h | 22M | 10,000 | 120,000 |
| Fig. 5(c): 20k | Human-only, 500h | 22M | 20,000 | 120,000 |
| Fig. 5(c): 30k | Human-only, 500h | 22M | 30,000 | 120,000 |
| Table 1: Multi-task Main Model | Human-only, 500h | 22M | 10,000 | 120,000 |
| Table 1: Single-task Main Models | Human-only, 500h | 22M | 10,000 | Single-arm: 30,000; Bimanual: 50,000 |
To ensure fair evaluation, we strictly controlled the experimental variables in both the data scaling experiments (Fig. 5a) and the model size scaling experiments (Fig. 5b), consistently using the encoder checkpoint at 10k updates and the DiT checkpoint at 120k updates for evaluation.
Based on these clarifications, we have explicitly distinguished these configurations in Appendix C and Appendix E, and provided a detailed configuration list in Appendix E.
Furthermore, we have conducted a systematic review of the other training and evaluation configurations reported in the manuscript, cross-checking them individually against the original configuration files, training logs, and actual model checkpoints used in the experiments. This ensures that the experimental settings reported in the manuscript are fully consistent with their corresponding implementations.
Response to Reviewer 4g9R(Part 3/3)
Additional Reproducibility Materials
To facilitate the verification of our implementation and experimental results, we have organized and provided the following anonymous materials:
-
Code: https://anonymous.4open.science/r/review-code-7f2a/
Includes the training, inference, and evaluation code, together with environment configurations and reproduction instructions.
-
Checkpoints: https://huggingface.co/modelarchivek7m29/model-checkpoints
Includes the weights of the main action representation encoder and its corresponding decoder, the multi-task main model, 11 single-task models, and three categories of ablation models (Raw Action, Action-MAE, and Action-iBOT). The corresponding configurations, model loading code, and evaluation instructions are also provided.
-
Project Page: https://anonymous.4open.science/w/arpg-review-homepage-D89E/
Includes demonstration videos of various real-world dexterous robotic manipulation tasks, an overview of the proposed method, and experimental results.
Once again, we sincerely thank the reviewer for the careful and constructive feedback. Your questions have helped us further improve the rigor of our experimental comparisons, the transparency of our training configurations, and the reproducibility of our work. We hope that the additional experiments and supplementary materials adequately address your concerns, and we would greatly appreciate any further comments or discussion.
Dear Reviewer 4g9R,
Thank you for your valuable feedback. We have carefully addressed your concerns in our rebuttal.
As the discussion period is coming to an end, we would greatly appreciate your thoughts on our responses. We would be happy to clarify any remaining concerns.
Thank you again for your time and consideration!
Response to Reviewer 4g9R(Part 1/3)
We sincerely thank the reviewer for the careful evaluation of our work and the constructive suggestions. Your concerns regarding the fairness of action execution protocols and the transparency of scaling experiments are particularly insightful, as these are precisely the aspects that were not sufficiently clarified in our original manuscript. To address these concerns, we have conducted additional experiments and thorough investigations, while also improving the accompanying reproducibility materials. Our detailed responses are provided below.
W1 & Q1: Fair Comparison under a Unified Action Execution Protocol
Yes, ARPG maintains a substantial advantage in bimanual manipulation even under matched execution protocols.
We fully agree with your concern that differences in execution protocols may introduce biases into performance comparisons. To thoroughly eliminate this potential confounding factor, we re-evaluated ARPG using exactly the same action-chunk execution mechanism as the DexJoCo baseline, while keeping the model weights, environments, and success criteria unchanged. As shown in the table below, ARPG continues to substantially outperform π0.5 on multi-task bimanual manipulation even under matched execution protocols. The additional experimental results are as follows:
These results further support our central argument that the advantage of ARPG fundamentally stems from action representation pretraining rather than differences in execution mechanisms.
Additionally, we observed an interesting phenomenon. Since ARPG performs inference in a pretrained latent space, the final action chunk is obtained by decoding the entire latent representation through the decoder. This means that actions at different timesteps within an action chunk, as well as actions corresponding to different components at the same timestep, are highly coupled. Consequently, the conventional prefix-attention-based RTC guidance mechanism, which operates in the raw action space, cannot guarantee smooth transitions between consecutive action chunks in this setting.
To ensure a rigorous evaluation, we re-evaluated all results using the revised action-chunk execution mechanism while keeping the original model weights unchanged. Accordingly, we have updated the results in Table 1 and Table 11 of Appendix H, and provided detailed descriptions of the execution protocols for each method in Appendix F.