- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents H2RB, a framework for adapting a robot-pretrained VLA from unpaired human and robot demonstrations. The method has two main components: (1) hand geometry-anchored alignment, which grounds human and robot observations through a shared hand-geometry target without paired data, and (2) a decoupled-decoder latent action tokenizer, which shares a latent action-prediction space while preserving embodiment-specific decoders for human and robot motion dynamics. The authors evaluate on a real OpenArm bimanual platform (using one arm/hand) on object transfer (Box/Ball/Towel pick-and-place introduced via human demonstrations) and skill transfer (air-fryer Open/Close absent from robot source data). H2RB substantially improves over shared-action baselines, e.g., object transfer average 51.8 vs. 27.2 for Ego-Pi-shared action, and skill transfer 62.7 vs. 0.0. Additional experiments study ID/OOD placement shift, heterogeneous mid-training, instruction conditioning, latent-gap analysis, and execution speed.
Critical strengths
Real-world non-paired human-to-robot transfer with a meaningful empirical gap. The paper demonstrates actual robot rollouts on object and skill transfer, and the gains over shared-action baselines are large, especially for skill transfer (0.0 → 62.7). This is consequential because it supports the central claim that sharing a latent action space while preserving embodiment-specific decoding can transfer human-demonstrated task semantics without paired human-robot data.
The method decomposition is conceptually sensible and partially supported by analysis. The separation between hand-geometry anchoring for perception and embodiment-specific decoding for action realization is a reasonable way to address the human-robot gap. The latent-gap reduction from 3.81 to 1.53 and the tokenizer/regression ablation are useful evidence that the learned latent space becomes more compatible across embodiments.
Critical weaknesses
The empirical evidence is too narrow and statistically weak for the strength of the claims. All real-robot results appear to come from a single robot platform, one arm/hand configuration, a small set of objects and skills, one independently trained policy per condition, and limited rollout counts without reported variance, confidence intervals, or significance tests. This is consequential because the paper claims general human-to-robot transfer and scaling to heterogeneous data, but the evidence does not establish robustness across seeds, tasks, embodiments, or evaluation conditions.
Baseline fairness and component attribution are not fully established. The main comparisons are mainly against HaWoR-shared action and Ego-Pi-shared action; stronger cross-embodiment or human-video baselines are not compared. The method combines several components—geometry anchoring, latent tokenizer, cross-domain regression, decoupled decoders, and instruction conditioning—and the ablation does not fully isolate their contributions across both object and skill transfer. In addition, H2RB’s object-transfer in-domain Doll PnP score is 49.0 versus 55.0 for the in-domain-only baseline, so the claim that in-domain robot performance is “preserved” needs clarification.
Questions for authors
In Table 1, H2RB obtains 49.0 on in-domain Doll PnP while the in-domain-only baseline obtains 55.0. Is this difference within noise, and how do you justify the claim that in-domain robot performance is preserved?
Were the HaWoR-shared-action and Ego-Pi-shared-action baselines tuned with the same backbone, data, compute, and hyperparameter search budget? Please clarify any baseline-specific tuning.
Can you provide a component-wise ablation isolating hand-geometry alignment, latent tokenizer, cross-domain regression, decoupled decoders, and instruction renewal on both object and skill transfer?
Why is the human decoder used at inference for the main transfer results? Is this required for deployment, and how sensitive are results to decoder selection?
4 comments
[W2-a, Q2] The fairness of shared-action baselines and the lack of broader cross-embodiment comparisons make the transfer improvements difficult to attribute.
Our shared-action comparisons already use the same robot-pretrained VLA backbone, human-robot training data, and 30k fine-tuning updates, reflecting that H2RB's gains are not attributable to differences in these conditions (Sec. 3.1; Appendix B.1). We also use the same hyperparameter-search budget across these methods. Importantly, H2RB itself relies on HaWoR-based hand reconstruction and Ego-Pi-style action retargeting, making the quality of these baseline components equally critical to our method.
- Broader baseline comparisons. Ego-Pi directly addresses human-to-robot adaptation of pretrained VLAs. To further broaden the evaluation, we additionally compare against HAT [1] and DexWild [2], retaining their original architectures, training recipes, and training scales while matching the human-robot data combinations and multi-task evaluation protocol. (Appendix Table 13)
| Method | Object transfer | Skill transfer |
|---|---|---|
| HaWoR-shared action | 10.7 | 0.0 |
| Ego-Pi-shared action | 27.2 | 0.0 |
| HAT-style adaptation | 15.6 | 0.0 |
| DexWild-style adaptation | 0.0 | 0.0 |
| H2RB | 51.8 | 62.7 |
Weighted task-progress scores (0-100). HAT/DexWild-style comparisons adapt their training principles.
Neither additional baseline surpasses Ego-Pi under our transfer protocol, while H2RB substantially outperforms all evaluated methods. These comparisons provide broader evidence for H2RB's transfer effectiveness beyond the original shared-action baselines.
[1] Qiu, Ri-Zhao, et al. "Humanoid policy~ human policy." arXiv preprint arXiv:2503.13441 (2025).
[2] Tao, Tony, et al. "Dexwild: Dexterous human interactions for in-the-wild robot policies." arXiv preprint arXiv:2505.07813 (2025).
[W2-b, Q3] The individual contributions of geometry alignment, latent action modeling, decoder architecture, and instruction conditioning are not sufficiently isolated across object and skill transfer.
We already demonstrated that geometry anchoring improves object transfer from 27.2 to 34.7, the latent tokenizer enables skill transfer at 49.7, and cross-domain regularization further improves it to 62.7 (Table 4a and Appendix Table 18). To further isolate their individual contributions, we additionally evaluate matched geometry-loss and decoder-architecture controls on both object and skill transfer. Instruction renewal is evaluated separately as an optional extension (Sec. 3.4). The staged tokenizer comparison captures the combined effect of latent-action modeling and decoupled decoding, whereas the controlled decoder comparisons isolate the architectural contribution.
Staged component ablation.
| Configuration | Object transfer | Skill transfer |
|---|---|---|
| Shared physical action | 27.2 | 0.0 |
| Geometry-anchored alignment | 34.7 | 0.0 |
| Latent tokenizer, without latent regularization | 43.3 | 49.7 |
| Cross-domain latent regularization (Full H2RB) | 51.8 | 62.7 |
Controlled component comparisons. (Appendix Table 19)
| Component | Configuration | Object | Skill |
|---|---|---|---|
| Geometry | H2RB w/o geometry loss | 49.7 | 57.7 |
| Geometry | H2RB w/ geometry loss | 51.8 | 62.7 |
| Decoder | Shared | 33.8 | 3.3 |
| Decoder | Source-conditioned | 38.9 | 11.7 |
| Decoder | Independent (H2RB) | 51.8 | 62.7 |
| Instruction | H2RB | 51.8 | 62.7 |
| Instruction | H2RB + Instruction renewal | 49.4 | 83.3 |
Object/skill values are weighted task-progress scores (0-100). Decoder comparisons use matched tokenizer and VLA training budgets.
[W2-c,Q1] The lower in-domain performance relative to robot-only fine-tuning raises concerns about whether H2RB preserves existing robot capabilities.
H2RB substantially mitigates in-domain performance degradation while acquiring new human-demonstrated behaviors, with only a 6-point decrease on Doll PnP relative to robot-only fine-tuning, compared with a 25-point decrease for Ego-Pi (Table 1). We clarify that capability preservation refers to retaining existing robot performance during human-to-robot adaptation.
While a performance gap remains on Doll PnP, H2RB substantially reduces the degradation observed with other human-robot adaptation baselines. On Box PnP, it also maintains performance comparable to robot-only fine-tuning (81.0 vs. 80.5), supporting its ability to retain existing robot capabilities while learning new target behaviors.
| Method | Object: Doll PnP (In-domain) | Skill: Box PnP (In-domain) |
|---|---|---|
| Robot-only FT | 55.0 | 80.5 |
| Naive co-training | 18.0 | 16.5 |
| HaWoR | 10.5 | 19.5 |
| Ego-Pi | 30.0 | 70.5 |
| H2RB | 49.0 | 81.0 |
Object/skill values are weighted task-progress scores (0-100).
[Q4] The necessity of human decoding at inference and its impact on object and skill transfer remain unclear.
We thank the reviewer for this question. We use the human decoder at inference because it better preserves the target motion patterns learned from human demonstrations. Both decoders produce robot-compatible wrist-hand actions, so human decoding is an empirical choice. The original object transfer decoder comparison in Appendix Table 21 shows higher performance with human decoding (51.8 vs. 32.7). We further evaluate skill transfer in Appendix Table 23, where the human decoder achieves 62.7 compared with 0.0 for the robot decoder. These results show that transfer performance is sensitive to decoder selection, supporting the use of the human decoder for human-demonstrated target tasks.
| Inference decoder | Object Avg. | Skill Avg. |
|---|---|---|
| Robot decoder | 32.7 | 0.0 |
| Human decoder | 51.8 | 62.7 |
Weighted task-progress scores (0-100).
Gentle Reminder
Dear Reviewer 3aEb,
We greatly appreciate your careful review and constructive feedback on our manuscript.
As the discussion period approaches its end, we wanted to follow up and ask whether you have any additional questions or concerns. We would be glad to provide further clarification.
Best regards,
Authors
[W1] The limited statistical reliability, task diversity, and embodiment coverage leave the robustness and scalability of H2RB insufficiently established.
H2RB already demonstrates full-task transfer, out-of-distribution generalization, and the benefits of heterogeneous mid-training in the original evaluation (Tables 3, 12; Sec. 3.3). We further strengthen this evidence through independent training seeds, additional target tasks, and a distinct robot embodiment.
[W1-a] Statistical reliability. The original evaluation already demonstrates improvements in strict task completion over Ego-Pi on both object (12.2% vs. 2.2%) and skill transfer (30.0% vs. 0.0%; Table 12).
To further assess reliability, we extend policy fine-tuning to independent training seeds while keeping the tokenizer fixed. We report seed-wise performance and task- and placement-stratified bootstrap confidence intervals, distinguishing rollout-level uncertainty from training-seed variability. Across all three seeds, H2RB consistently outperforms the originally reported Ego-Pi baseline in strict task completion, providing further evidence that the transfer gains are not specific to the original fine-tuning seed. We have added the seed-wise results and bootstrap confidence intervals to Appendix Table 15.
[W1-b] Evaluation diversity and generalization. We already demonstrated generalization to unseen object placements, improving out-of-distribution transfer from 21.3 to 45.3 over Ego-Pi (Table 3). We next extend evaluation beyond the original Box/Ball/Towel PnP and Open/Close tasks to Coffee PnP and Pour Coffee, while also testing changes in workspace geometry. H2RB consistently outperforms the evaluated baselines across these additional settings. We have added the new task and table-height comparisons to Appendix Tables 16 and 17, with the data collection setup in Appendix Figure 9.
Weighted task-progress scores (0-100). The table-height shift (79 -> 89 cm) changes camera-to-workspace geometry and reachability.
We additionally evaluate visual background variations by replacing the original tabletop covering with a checkered tablecloth. H2RB achieves 47.3 compared with 51.8 under nominal conditions, showing a 4.5-point decrease under this appearance shift. Together, these additional evaluations broaden the evidence for H2RB beyond the original tasks and evaluation conditions, demonstrating effective transfer to new manipulation behaviors and resilience to the tested geometric and visual appearance changes. We have added the quantitative background shift results to Appendix Table 17(b) and the qualitative evaluation sequences to Appendix Figure 10.
[W1-c] Heterogeneous scaling across embodiments. We already demonstrated that heterogeneous human-robot mid-training improves object and skill transfer on OpenArm (Sec. 3.3). We further confirm that these gains extend to full-task completion, with strict success increasing from 12.2% to 21.1% for object transfer and from 30.0% to 56.7% for skill transfer. We have added the MT completion results to Appendix Table 28 and progress results to Appendix Table 12.
We additionally evaluate RB-Y1, a distinct bimanual robot with a 58-dimensional action interface. Human-robot MT improves parcel-handling success from 64% to 72%, reaching 78% with geometry anchoring.
50 parcel-orientation trials per condition.
Unlike the OpenArm task transfer, the RB-Y1 experiment evaluates heterogeneous mid-training followed by robot-only fine-tuning, without the shared action tokenizer. This complementary result indicates that human-derived representations can also benefit adaptation to a different robot embodiment, beyond the original OpenArm setting. We have added the RB-Y1 evaluation result to Appendix Table 29 and Figure 12.