- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents PhysV2V, a framework for editing object physics in videos. Given an input video, it reconstructs object geometry, initializes physical properties using a vision-language model (VLM), and optimizes these properties by matching projected simulation trajectories to tracked video points. Rigid and deformable objects are handled using Brax and the material point method (MPM) implementation from PAC-NeRF, respectively. After user-specified parameter changes, the scene is simulated again, and the resulting point trajectories condition a frozen LTX-Video backbone adapted with IC-LoRA. The evaluation includes synthetic physical edits, qualitative real-video examples, and ablations of geometry estimation and physical-property optimization.
Critical strengths
-
The problem is relevant, and using source-video motion to support physical editing is well motivated. Fitting an editable physical proxy to observed motion provides a useful way to connect the dynamics of the source video with subsequent parameter changes. This addresses an important limitation of approaches that initialize a scene from a single image.
-
The experiments provide initial evidence for the usefulness of per-video physical-property optimization. The comparisons and ablations show improvements in output-video similarity, including over a variant without property optimization that receives ground-truth initial velocities. The real-video examples also illustrate edits to rigid-body motion and elastic deformation.
Critical weaknesses
- The scope of compatibility with rigid-body and continuum dynamics is unclear.
The paper claims to design a per-scene property optimization framework that is “compatible with both rigid-body and continuum dynamics.” However, it does not explain whether this means supporting the two object types separately or handling scenes containing interacting rigid and deformable objects.
Section 4.2 uses Brax for rigid bodies and the PAC-NeRF MPM simulator for deformable objects, while the method and appendix describe these simulations separately. Joint simulation would require explicit handling of contact, momentum exchange, time stepping, and gradient propagation between the solvers; their individual differentiability does not automatically establish such a coupled system. No coupling mechanism or mixed-interaction experiment is provided. The authors should clarify the intended scope and, if mixed scenes are supported, describe and validate the interaction mechanism.
- The evaluation provides insufficient direct evidence for reliable physical-parameter editing across scenes.
The main quantitative experiments in Tables 2 and 3 evaluate the final videos using PSNR, SSIM, and LPIPS. These comparisons provide evidence about edited-video similarity, but do not directly quantify parameter-estimation accuracy or how accurately the generated motion realizes the requested physical intervention. Moreover, the synthetic evaluation consists of parameter variations of a single two-ball collision setup.
Direct parameter-level evidence is mainly in Appendix A.2: Figure 7 shows optimization curves for one example, while Table 5 does not specify the dataset, number of scenes or samples, repeated runs, or aggregation protocol. Table 5 also initializes parameters at scaled ground-truth values, rather than evaluating the default VLM initialization across diverse inputs.
Although exact parameter recovery is not the authors’ stated objective, the paper should more directly establish that the inferred proxy supports accurate physical edits. Quantitative trajectory and physical-response comparisons before and after edits, across multiple independent scenes and edit magnitudes, would better support this claim. The issue is the limited scope and specificity of the evidence, rather than the placement of results in the appendix itself.
- The methodological novelty appears limited.
The reconstruction, tracking, simulation, and generation stages rely on existing components. The generation stage adopts existing IC-LoRA adapters. The paper’s main contribution appears to be the integration of video-based parameter fitting, physical interventions, and conditional video generation. The novelty of this integration is insufficient.
Questions for authors
-
Does the framework support scenes containing interacting rigid and deformable objects? If so, how are contact, momentum exchange, time stepping, and gradient propagation handled between Brax and MPM?
-
What dataset, scenes, and sample counts are used in Table 5?
-
Can the authors quantify trajectory and physical-response errors under different edit magnitudes across multiple scenes, and evaluate these errors for both the simulator output and the final generated video?
-
What specific methodological advance is introduced beyond combining existing reconstruction, tracking, simulation, and IC-LoRA-based generation components? Which controlled comparisons establish the benefit of the proposed design choices?