- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies interaction-preserving video reshooting. Given a source video and a target camera trajectory, it generates novel views that preserve object-hand contact and motion. They use image-to-video (I2V) generation models as a data source. The authors build ObjAlive, a collection of 13.5K filtered I2V-generated videos covering contact-rich manipulation, articulation, deformation, fluids, topology, and appearance changes. They turn these single-view clips into multi-view training pairs via double reprojection, mix them with synchronized multi-view and other real single-view videos, and fine-tune Wan2.1-I2V-14B conditioned on the source video and a target-view warp, with camera-ray and coordinate encodings injected into the DiT attention. Validation relies mainly on a blinded human preference study (21 participants), where ROVER receives 71.6% and 63.6% of interaction-fidelity votes on human-object and robotic manipulation videos, respectively, plus the best results on DyCheck. They also conduct a step-matched ablation on 10 videos (69.0% preference for training with ObjAlive), and qualitative long-video and 4D reconstruction demonstrations are also provided.
Critical strengths
-
Synchronized multi-view data rich in contact and object-state changes is scarce, and this hinders reshooting methods for interactions. Turning generated single-view clips into training pairs via double reprojection avoids expensive synchronized capture. In the step-matched ablation, the model trained with ObjAlive is clearly preferred on all three criteria against the variant without it. Other reshooting methods could reuse such data.
-
ROVER leads by significant margins in both human-object and robotic settings and on all three criteria, and performs well on DyCheck, which comes with multi-view ground truth. The main model is trained with no robot data yet is capable on robotic videos. The paper also reports failure cases, inference cost, and limitations.
Critical weaknesses
- The technical contribution seems limited, and its significance is unclear. The main new element of the paper is ObjAlive, while the remaining parts of the training method (double reprojection, Plücker-ray camera encoding and injection, etc.) are largely existing techniques. The main contribution therefore seems reduced to the ObjAlive construction pipeline. However, this pipeline is a common recipe for synthesizing training data with a video generator (VLM-written prompts, video diffusion model, filtering, etc.), and the paper does not point to technical designs in it that are specific or new. Overall, this amounts to a fairly standard distillation procedure.
More importantly, the paper does not convincingly show that the distillation is meaningful and necessary. ObjAlive is generated by Wan2.2-I2V-14B, while ROVER is fine-tuned from the earlier Wan2.1-I2V-14B. Would a model that uses Wan2.2 directly as the base, trained with the same recipe but without the filtered ObjAlive data, perform comparably or even better than ROVER?
- The central claim is insufficiently supported. The paper's main claim is that ObjAlive can distill a video generator's interaction priors into the reshooting model. However, the experiment cannot separate interaction priors from having more single-view dynamic data. The variant without ObjAlive loses about 1/3 of the training scenes (the 13.5K ObjAlive clips in Table 2 are about 1/3 of all scenes and about 4/5 of the single-view data) while keeping the same number of training steps, so differences in data quantity and diversity are mixed in. The key missing controls are an equally sized set of I2V videos generated with prompts unrelated to interactions, and an equally sized set of real single-view videos. In addition, the only quantitative evaluation, DyCheck, is not an interaction dataset and has no ObjAlive ablation, so it cannot support the claim quantitatively either.
Moreover, the data are generated from the same model family (ObjAlive is generated by Wan2.2-I2V-14B, and ROVER is fine-tuned from Wan2.1-I2V-14B). Fine-tuning on videos from the same family may be closer to the data distribution/manifold the base model already knows (e.g., visual style, motion statistics, temporal smoothness), making fine-tuning less likely to drift from its original capabilities. Such a benefit can be mixed with the interaction prior.
Questions for authors
-
Do all baselines use the same DA3 depth and cameras? Resolutions differ across methods in Table 4 (672×384, 832×480, 1280×704); were resolution and frame count equalized in the human study? Which CFG setting was used in the main results and the human study, and did baselines receive comparable guidance tuning? How were the 25 evaluation videos selected (also, 25 videos seem not comprehensive enough)?
-
For ObjAlive, what are the total generated count, filter pass rate, and the effect of filtering? Generating high-quality videos with rich hand-object interaction seems to remain challenging for current video generation models
-
The robotic fine-tuning uses only 400 steps and is evaluated on 10 videos. Can this rule out under-training or low statistical power?
The reviewer is willing to raise the score if the concerns can be addressed.