- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
RoboEdit proposes a method for ref-guided editing of robotic manipulation videos. This is to replace a robot's appearance while preserving the surrounding scene and motion. It fine-tunes HunyuanCustom using self-reconstruction examples built from existing videos, to avoid paired recordings across robot embodiments. Its core contribution is a token-correlation loss that compares ref-conditioned and null-ref passes and discourages changes in rel between tokens inside and outside the mask. Token sampling reduces the correlation computation. Recon and temporal losses provide extra supervision.
The paper evaluates the method on 335 video-ref pairs using visual metrics, a new robot manipulation consistency metric, human pref, and ablations. It also reports improved downstream control after pretraining visual representations on the output videos (average sim success reaches 41.1% compared to 33.2% for the strongest reported baseline). Some OpenArm experiments show gains too.
Critical strengths
- The edited videos improve downstream control in simulation. Across 9 tasks and 3 robots, it achieves a higher average success than the strongest baseline that is reported. So seems like there is good evidence that the output videos have practical value for rep learning.
- Ablation study is good and supports the claims... Adding the token-correlation loss improved DINO similarity and the scene-preservation judge scores. The remaining training setup is unchanged and so the comparison supports that the proposed loss is meaningful.
- It addresses a data bottleneck and the setup is useful practically speaking. It builds supervision from existing videos using extracted masks and reference crops. So it avoids collecting matching demonstrations that repeat the same scene and trajectory with different robots (actual paired data).
- Specifying the target robot through an image seems convenient. The method accepts a ref image and an edit mask without requiring a separate translator for every source-target robot pair or a calibrated simulator model of the target robot. This makes the setup attractive when suitable erndering assets or target domain recordings are unavailable. Generalization ot unseen robot identities still needs a clearer eval.
Critical weaknesses
- It is not fully supported that the model can generalize to other unseen embodiments. The benchmark is described as OOD, but its separation from training data is not specified by dataset, trajectory, scene, task, or robot identity. It is especially unclear which target reference robot were absent during finetuning. This happens to be a central claim about handling new embodiments and is not fully supported by the experiments.
- RMC doesn't directly establish that maniuplation remains correct. The metric combines mask overlap, robot-mask centroid agreement, temporal feature similarity, and ref matching. These can miss imperfect contact, object motion, or end-effector trajectories (or at least it is unclear how it won't miss). This matters because the manipulated object lies inside the regenerated region.
- The main quantitative results are not extensive. Table 1 includes only HunyuanCustom, and it is unclear whether the baseline received comparable robot data finetuning. UniVideo and Wan 2.2 replace are excluded because of high failure rates without reporting rates. The controlled ablation supports the regularizer, but the broader comparison against existing editing methods is not fully conclusive.
- The submission is not accompanied by video results. This is a highly visual method and it is close to impossible to actually validate the claims without being able to look at visual results.
Questions for authors
Clarification on the weaknesses mentioned above.
3 comments
Thanks for the detailed rebuttal and more experiments. The new results, especially the object-motion eval and regularizer comps, strengthen the paper and address most of my initial concerns. Some limitations remain, especially around generalization, but I don't see them as major blockers. Overall, I'm satisfied with the rebuttal and will maintain my weak accept rating.
We thank the reviewer for the positive feedback and for acknowledging the additional experiments. We appreciate your constructive suggestions, which have helped us strengthen the paper.
Response to W1: Generalization to Unseen Robot Embodiments
We thank the reviewer for pointing out the need to clarify our out-of-distribution evaluation setting.
First, our evaluation benchmark is disjoint from the finetuning data. RoboEdit is finetuned on 57,343 real-world manipulation videos from seven OXE datasets. The evaluation benchmark contains 335 video-reference pairs from four separate sources (Jaco, KUKA, miscellaneous web videos, and self-collected laboratory videos), with no overlapping training episodes or trajectories.
Second, the benchmark includes both seen and unseen target robot embodiments. While some references correspond to robot families present during finetuning (e.g., KUKA, UR, and FANUC), others represent unseen robot types, including RealMan, Elephant, and Kinova. RoboEdit performs these replacements without target-specific finetuning or paired cross-embodiment demonstrations.
Third, our downstream simulation experiments provide additional evidence of cross-domain transfer. RoboEdit is finetuned entirely on real-world videos, while the downstream experiments use simulated Panda robots with appearances and rendering conditions absent from the finetuning data.
We have clarified the dataset splits, reference identities, and generalization settings in the revised manuscript: Section 5.3 now describes the held-out evaluation sources and the seen vs. unseen target references, and Appendix E ("Detailed Experimental Settings") adds an Evaluation Split and Unseen References paragraph detailing the training/evaluation split and the seen vs. unseen reference robots.
Response to W2: Object Motion and Interaction Preservation
We thank the reviewer for raising this concern. We agree that RMC primarily measures visual consistency and may miss errors in object motion or physical contact.
To address this, we conduct an additional evaluation in simulation, where ground-truth object trajectories are available. For each frame, we use the recorded 3D object position and camera parameters to determine where the manipulated object should appear. We then detect its position in the edited video and compare it with the corresponding GT-rendered video. Frames with unreliable GT detections are excluded.
With token-correlation regularization, object position error decreases by 35.4%, while detection rate improves by 10.7 percentage points. Unlike RMC's robot-mask centroid metric, this evaluation directly measures whether the manipulated object remains at its expected visual position.
Together with our downstream control ablations, these results provide additional evidence that the regularizer improves manipulation-relevant motion preservation. Although physical contact and end-effector accuracy are not directly measured, the new evaluation addresses object-motion errors that RMC may overlook (Appendix V).
Response to W3: Additional Quantitative Comparisons and Regularizer Ablations
We thank the reviewer for this suggestion. We have expanded the quantitative evaluation with additional editing baselines and regularizer comparisons.
Additional editing baselines (Appendix T). We quantify three failure modes of UniVideo and Wan2.2-Replace: original-arm residuals, unsuccessful replacement, and motion mismatch. Only 3.3% of UniVideo edits and 6.7% of Wan2.2-Replace edits pass all three checks in the sampled evaluation, explaining their exclusion from the main quantitative comparison.
Additional regularizer comparisons (Appendices S and U). We compare our token-correlation regularizer against two alternatives under matched training settings:
Downstream regularizer ablations (Appendices M and R). Removing the regularizer reduces frozen-encoder BC success by 9.5–13.4 percentage points across three source embodiments. In end-to-end π0.5 fine-tuning, target-domain success decreases from 43.0% to 29.0% without the regularizer.
These additional experiments strengthen the quantitative comparison and demonstrate that our regularizer improves both local scene preservation and downstream robot learning.
Response to W4: Supplementary Videos
We thank the reviewer for the suggestion. We have provided separate video files in the supplementary materials. We hope the new evidence adequately addresses the concerns and warrants a reassessment of the rating.