- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces HumanVerse-500, a 500-hour egocentric dataset containing synchronized whole-body and hand motion, and λ0, a humanoid VLA policy trained through three stages of human and robot data. The framework transfers human demonstrations to humanoid control through shared model representations and embodiment-specific action interfaces. Experiments demonstrate 90.0% success on SIMPLE and 62.5% success across four real-world tasks, with additional ablations showing benefits from whole-body human-data pretraining.
Critical strengths
-
HumanVerse-500 provides 500 hours of diverse synchronized egocentric and whole-body demonstrations, offering valuable supervision for humanoid loco-manipulation.
-
The method achieves promising simulation and real-world performance, with training-stage ablations and data-scaling experiments demonstrating the benefits of whole-body human data.
Critical weaknesses
-
Appendix D indicates that the SIMPLE results use task-specific best configurations without establishing validation-only selection, raising concerns about the fairness of the reported comparisons.
-
The experiments focus on G1 and use single-task training for SIMPLE, with limited real-world trials to establish robustness.
Questions for authors
-
Can the authors provide SIMPLE results using checkpoint and inference configurations selected exclusively on validation data under a consistent evaluation protocol?
-
Can the authors provide additional ablations isolating the contributions of whole-body motion supervision, hand motion, and the SONIC-based action representation?
-
Can the authors evaluate a unified multi-task policy or provide additional trials and training seeds to substantiate the generalization claims?
2 comments
Weakness 2 & Question 3:
Manuscript context: and . The shared-policy results and videos are provided in this response only.
- One checkpoint across all six SIMPLE tasks. We report a policy jointly trained on six tasks for 100,000 steps. Every task, including XMovePick, uses the same final checkpoint, without task-specific fine-tuning or switching model weights. Task-specific normalization statistics are retained.
| Task | Successful trials | Success |
|---|---|---|
| XMovePick | 23/30 | 76.7% |
| BendPick | 30/30 | 100.0% |
| Handover | 22/30 | 73.3% |
| Mobile pick-and-place | 12/30 | 40.0% |
| Grasp | 28/30 | 93.3% |
| XMoveBendPick | 25/30 | 83.3% |
| Overall | 140/180 | 77.8% |
-
What can be compared. Each task includes ten trials at each of three difficulty levels. These saved runs precede later changes to reset, warmup, and time-budget handling. Together with the different training budget, this means 77.8% is not a controlled comparison against the 90.0% single-task result. It provides evidence that one set of model weights can perform all six tasks.
-
Real-world multi-task evidence. A separate policy jointly trained on cart pushing, fruit delivery, and takeout bag disposal performs all three tasks with the same weights. We provide as qualitative evidence, alongside the six-task SIMPLE demonstrations. This is a different model from the six-task SIMPLE policy, and the videos are not a repeated-trial success-rate evaluation.
Response to Reviewer NmFZ
Thank you for your detailed and constructive review, and for recognizing the value of synchronized whole-body human data. We especially appreciate your careful attention to checkpoint selection and evaluation fairness. You identified wording that could reasonably suggest test-set-based tuning, and we are grateful for the opportunity to correct it and make the actual protocol explicit. Your suggestions on supervision and multi-task evaluation also motivated us to provide the hand-only Stage-II comparison and report the available shared-policy results. We distinguish these additional findings from the robustness evidence that would require further trials or independent training seeds. The protocol clarification is highlighted in red in the revised manuscript. The additional hand-only ablation and shared-policy results are reported below and on the response webpage.
Please see our response webpage for detailed results and videos.
Weakness 1 & Question 1: Checkpoint selection and evaluation fairness
Revised manuscript: Single-task training and evaluation protocol, Appendix D, p. 26.
A fixed protocol, not test-set selection. Thank you for catching the misleading wording in Appendix D. We apologize for not describing the procedure accurately. For the reported single-task SIMPLE experiments, each task-specific policy is post-trained for 40,000 steps using the same training configuration. We evaluate the final 40,000-step checkpoint with a fixed inference configuration. We do not select checkpoints or tune inference settings according to test scores.
Training configuration and correction. The setup is informed by the SIMPLE baselines and Ψ₀. The per-task 40,000-step budget follows Ψ₀, Sec. IV-A.3, p. 6. We have corrected the statement about selecting the “best recorded configuration separately for each task” in Appendix D, p. 26 of the revised manuscript. The results were obtained with the fixed protocol above, not a task-wise search for the best test score. Thus, the correction concerns our description, rather than a change in checkpoint-selection procedure.
Question 2: Whole-body supervision and action-representation ablations
Manuscript context: Training-stage ablations, Table 3, p. 9 and Appendix G, p. 28. The additional hand-only comparison is provided in this response only.