acceptodds
Under review as a conference paper at ICLR 2027

Rover: Interaction-Preserving Video Reshooting

Abstract

Extending video reshooting from general scenes to object interactions requires preserving contact, motion, and object state across viewpoints. However, existing synchronized multi-view datasets offer limited coverage of interaction dynamics under diverse camera trajectories. We present Rover for interaction-preserving video reshooting. Our approach distills interaction and dynamics priors from a pretrained video generator through ObjAlive, a collection of 13.5K filtered videos produced via image-to-video generation. The collection covers contact-rich manipulation, fluid motion, deformation, topology changes, articulation, and appearance changes. We combine it with synchronized multi-view and real single-view videos to fine-tune a video diffusion model conditioned on source videos and target-view warps. Blinded four-way human evaluation favors Rover, which receives 71.6% and 63.6% of interaction-fidelity votes on held-out human–object interaction and robotic manipulation videos, respectively. These results indicate generalization to robotic manipulation without robot-specific fine-tuning. A controlled, step-matched ablation demonstrates the benefit of ObjAlive for preserving contact and non-rigid dynamics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video generation, camera control, interaction-centric

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_v1YXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies interaction-preserving video reshooting. Given a source video and a target camera trajectory, it generates novel views that preserve object-hand contact and motion. They use image-to-video (I2V) generation models as a data source. The authors build ObjAlive, a collection of 13.5K filtered I2V-generated videos covering contact-rich manipulation, articulation, deformation, fluids, topology, and appearance changes. They turn these single-view clips into multi-view training pairs via double reprojection, mix them with synchronized multi-view and other real single-view videos, and fine-tune Wan2.1-I2V-14B conditioned on the source video and a target-view warp, with camera-ray and coordinate encodings injected into the DiT attention. Validation relies mainly on a blinded human preference study (21 participants), where ROVER receives 71.6% and 63.6% of interaction-fidelity votes on human-object and robotic manipulation videos, respectively, plus the best results on DyCheck. They also conduct a step-matched ablation on 10 videos (69.0% preference for training with ObjAlive), and qualitative long-video and 4D reconstruction demonstrations are also provided.

Critical strengths
  1. Synchronized multi-view data rich in contact and object-state changes is scarce, and this hinders reshooting methods for interactions. Turning generated single-view clips into training pairs via double reprojection avoids expensive synchronized capture. In the step-matched ablation, the model trained with ObjAlive is clearly preferred on all three criteria against the variant without it. Other reshooting methods could reuse such data.

  2. ROVER leads by significant margins in both human-object and robotic settings and on all three criteria, and performs well on DyCheck, which comes with multi-view ground truth. The main model is trained with no robot data yet is capable on robotic videos. The paper also reports failure cases, inference cost, and limitations.

Critical weaknesses
  1. The technical contribution seems limited, and its significance is unclear. The main new element of the paper is ObjAlive, while the remaining parts of the training method (double reprojection, Plücker-ray camera encoding and injection, etc.) are largely existing techniques. The main contribution therefore seems reduced to the ObjAlive construction pipeline. However, this pipeline is a common recipe for synthesizing training data with a video generator (VLM-written prompts, video diffusion model, filtering, etc.), and the paper does not point to technical designs in it that are specific or new. Overall, this amounts to a fairly standard distillation procedure.

More importantly, the paper does not convincingly show that the distillation is meaningful and necessary. ObjAlive is generated by Wan2.2-I2V-14B, while ROVER is fine-tuned from the earlier Wan2.1-I2V-14B. Would a model that uses Wan2.2 directly as the base, trained with the same recipe but without the filtered ObjAlive data, perform comparably or even better than ROVER?

  1. The central claim is insufficiently supported. The paper's main claim is that ObjAlive can distill a video generator's interaction priors into the reshooting model. However, the experiment cannot separate interaction priors from having more single-view dynamic data. The variant without ObjAlive loses about 1/3 of the training scenes (the 13.5K ObjAlive clips in Table 2 are about 1/3 of all scenes and about 4/5 of the single-view data) while keeping the same number of training steps, so differences in data quantity and diversity are mixed in. The key missing controls are an equally sized set of I2V videos generated with prompts unrelated to interactions, and an equally sized set of real single-view videos. In addition, the only quantitative evaluation, DyCheck, is not an interaction dataset and has no ObjAlive ablation, so it cannot support the claim quantitatively either.

Moreover, the data are generated from the same model family (ObjAlive is generated by Wan2.2-I2V-14B, and ROVER is fine-tuned from Wan2.1-I2V-14B). Fine-tuning on videos from the same family may be closer to the data distribution/manifold the base model already knows (e.g., visual style, motion statistics, temporal smoothness), making fine-tuning less likely to drift from its original capabilities. Such a benefit can be mixed with the interaction prior.

Questions for authors
  1. Do all baselines use the same DA3 depth and cameras? Resolutions differ across methods in Table 4 (672×384, 832×480, 1280×704); were resolution and frame count equalized in the human study? Which CFG setting was used in the main results and the human study, and did baselines receive comparable guidance tuning? How were the 25 evaluation videos selected (also, 25 videos seem not comprehensive enough)?

  2. For ObjAlive, what are the total generated count, filter pass rate, and the effect of filtering? Generating high-quality videos with rich hand-object interaction seems to remain challenging for current video generation models

  3. The robotic fine-tuning uses only 400 steps and is evaluated on 10 videos. Can this rule out under-training or low statistical power?

The reviewer is willing to raise the score if the concerns can be addressed.

Reviewer_uw9aOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

ROVER synthesizes novel-view videos while preserving source interactions. Its principal contribution is OBJALIVE, 13.5K filtered generated videos used alongside synchronized multi-view and real single-view data, with geometric attention conditioning and a long-video extension.

Critical strengths
  1. The generated-data approach addresses scarce synchronized interaction videos across diverse dynamics.
  2. ROVER receives 71.6% and 63.6% of interaction-fidelity votes and leads all six reported DyCheck metrics.
  3. The OBJALIVE ablation controls training steps, architecture, initialization, and other data sources, while the appendix documents failures and limitations.
Critical weaknesses
  1. Beyond OBJALIVE, the methodological advance appears incremental, combining geometry-conditioned diffusion established by ViewCrafter, double reprojection from TrajectoryCrafter, and Wan fine-tuning, leaving the distinct methodological contribution insufficiently justified.
  2. Generalization to large viewpoint changes remains insufficiently established because qualitative examples are not supported by systematic evaluation across camera rotation, translation, and increasing disocclusion.
  3. Quantitative evaluation relies on image similarity and human preferences, while the 4D reconstruction results remain qualitative and lack metrics for 3D geometry, cross-view consistency, and contact preservation.
Questions for authors

Although evaluation inputs are matched, scenes, view selection, temporal sampling, and aggregation weights are unspecified.

Reviewer_mwFpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies interaction-preserving video reshooting: given a source video and a target camera trajectory, it synthesizes new views that preserve contact, motion, deformation, and topology changes. The key observation is that interaction/dynamics priors can be distilled from a pretrained video generator without synchronized multi-view capture

Critical strengths

Distilling interaction priors from a frozen I2V generator and training via single-view reconstruction instead of collecting synchronized multi-view is a meaningful consequence.

History-conditioned windowed generation with 3D background memory yields consistent 259-frame reshoots, and the 4D-reconstruction application demonstrates downstream utility rather than only visual demos.

Critical weaknesses

Evaluation rests almost entirely on subjective preferences with small samples. 25 raters × 25 clips per domain is thin, and the main text offers no objective geometry/contact/camera-trajectory metrics (DyDCheck is in the appendix). Without ground-truth contact measurement, the central "preserves interaction" claim is weakly verified.

Synthetic-prior circularity and baseline fairness. OBJALIVE is produced and filtered by the same family of generative/VLM models (Wan I2V + Qwen3-VL), so ROVER may inherit the generator's failure modes rather than learn real interaction physics; the reconstruction target is itself a depth-derived warp that can be wrong at contact. It is also unclear whether EX-4D/GEN3C/Vista4D were tuned on comparable data or are off-the-shelf deterministic warps, which could inflate the margins.

Questions for authors

see weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.