- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a human-to-robot video editing suite consisting of an automatic geometry/retargeting pipeline for constructing paired human–robot videos, a learned cross-embodiment video editor trained on these pairs, and a 3D robot-state decoder. The resulting dataset contains 14.1M paired frames across seven embodiments. The method is evaluated on video-editing metrics, robot-state reconstruction, and trajectory-conditioned downstream control.
Critical strengths
Large-scale and coherent system contribution. The paper integrates automatic human-object reconstruction, embodiment retargeting, video editing, and robot-state decoding into a unified pipeline and demonstrates it at substantial scale across multiple robot embodiments. Although the individual components are largely based on existing techniques, the resulting system enables a form of paired human–robot supervision that would otherwise be expensive to obtain. I view this system- and data-level capability as a substantive contribution independent of architectural novelty.
Potentially useful amortization of expensive geometric supervision. The paper introduces RoboEdit-Trans, a learned robot-conditioned video editor that amortizes part of the expensive per-video reconstruction, retargeting, physics refinement, and compositing pipeline used by RoboEdit-ADC. Once trained on the automatically curated paired data, RoboEdit-Trans can translate new human manipulation videos through model inference rather than repeatedly executing the full geometric pipeline for every example. I view this amortization of an expensive data-generation procedure into a reusable learned model as an important practical contribution toward scaling human-to-robot video supervision.
Critical weaknesses
Scalability beyond the current source distributions is not yet established.
While the paper demonstrates impressive scale in the amount of generated data, it remains unclear whether this scalability extends to substantially more unconstrained in-the-wild human videos. This limits how strongly I can interpret the broader claim of unlocking abundant human-video supervision.
The practical robotics utility of the recovered trajectories remains insufficiently characterized.
The real-robot experiments demonstrate that the recovered trajectories can provide useful references for downstream control, but the current evaluation does not clearly establish what level of robot-relevant information they reliably preserve. This makes the practical scope of the structured trajectory supervision difficult to assess.
Questions for authors
Scalability to more unconstrained human videos.
A central motivation of the paper is to leverage abundant human manipulation videos at scale. Could the authors clarify how well the proposed framework extends beyond the source distributions considered in the current experiments to more unconstrained in-the-wild videos? In particular, I would be interested in both (i) the robustness of the automatic data-construction pipeline under substantially greater variation in viewpoint, occlusion, scene complexity, objects, and interactions, and (ii) the generalization of the learned RoboEdit-Trans model to such inputs. Evidence or analysis along either of these axes would help better establish the practical scalability of the approach.
Robotic utility of the recovered trajectories.
I would appreciate a clearer characterization of the level of robotic supervision provided by the recovered trajectories at their current accuracy. The real-robot experiments show that these trajectories can serve as useful references for a downstream residual controller, but this leaves somewhat open how the trajectories themselves should be interpreted from a robot-learning perspective. Are they intended to function as approximately executable demonstrations, coarse motion references that require substantial corrective control, or structured supervision for higher-level aspects of the interaction? Any additional evidence or analysis characterizing which aspects of the recovered motion are reliable enough for downstream use would help clarify the practical scope of this contribution.