- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper focuses on physical counterfactual scenarios in video editing, where an edit should change not only the visual content but also the subsequent motion and interactions in the scene. In this sense, the task also evaluates whether existing video generation and editing models have meaningful physical priors. The authors propose VideoPhysEdit, a training-free pipeline that explicitly reconstructs a physical scene, applies the requested physical edit, and uses the simulated results to guide video generation. The paper also introduces a dedicated benchmark for this task. The experiments show that existing open-source and commercial models generally perform poorly on this challenging setting, while VideoPhysEdit achieves much better physical edit accuracy.
Critical strengths
One critical strength is that the carefully designed pipeline is effective both in theory and in practice. It essentially lets the generation model focus only on rendering, while using the reconstructed physical environment for reasoning and simulation. Its performance is therefore clearly better than that of the best current generation models. When the prompt is not sufficiently detailed, current video generation models do not have strong physical reasoning ability. They lack too much context to understand the physical parameters and perform counterfactual reasoning.
Critical weaknesses
-
The method is not very novel, as explicit reconstruction is already relatively common in the video generation field. A pipeline that reconstructs a physical environment and then performs reasoning will also naturally be more accurate than implicit generation.
-
Ideally, generation models should learn general physical laws and reasoning abilities about the physical world. This approach does not seem very scalable or generalizable, and it appears to work only for specific physical scenarios.
Questions for authors
-
The full pipeline takes much longer than a single video generation pass. With more streaming video generation models emerging, do the authors think this approach could be adapted to low-latency or interactive settings?
-
Is the comparison with other generation models fully fair when they only receive the source video and a short counterfactual editing instruction, while VideoPhysEdit explicitly reconstructs the physical environment before reasoning? The other models may lack enough scene-specific context to infer the physical parameters and downstream effects. Appendix C.5 partly studies this issue on four tasks, but could the authors provide broader evidence or discussion about this potential input-context disadvantage?
5 comments
Thanks to the authors for their detailed response and efforts. While I appreciate the clarifications, my concerns about the novelty and generalizability to complex real-world physical scenarios remain.
Thank you for your review, and we respect any decision you make.
We would still like to reiterate that physical editing in complex real-world scenes is also the goal we pursue, yet even the most advanced current models completely fail on the simplest scenes, whereas our method at least already takes a big step forward. In addition, regarding novelty, we emphasize again that we are the first to explicitly reconstruct kinematics with collision interactions, that every stage is our new design with extensive theoretical and experimental support, and that the design of each stage addresses a challenging problem in 3D.
Finally, thank you again for your review.
Thank you for your careful review and recognition of the value of separating physical reasoning from visual generation. We address two weaknesses and two questions.
W1. Methodological novelty.
We agree reconstruction has precedents. We introduce physical counterfactual video editing (PCVE), a new task, a new editing framework and dynamics-prediction pipeline, as no ready-made solution directly covers source-motion reproduction and post-intervention physical rollouts. Stages 2–7 each have a new task-specific design: Motion Observation Analysis (2), Canonical and Anchor Scene Reconstruction (3), Motion Prior Reconstruction (4), Physical Inversion (5), Physical Intervention (6), and Counterfactual Video Generation (7). We also introduce PCVE-RigidBench, a new benchmark with paired targets and physical ground truth, and PES for physical evaluation.
W2. Scope and scalability.
VideoPhysEdit covers falling, sliding, rolling, bouncing, collisions and collision chains, with edits to mass, friction, restitution, velocity and object addition/removal. Fixed cameras, planar supports and contacts adequately approximated by sphere/box proxies define a reusable family of rigid-body systems.
These camera, geometry and motion settings have broad precedents:
Generation: Static cameras: PISA (ICML 2025), PhysStream (SIGGRAPH Asia 2026), Generating Videos with Scene Dynamics (NeurIPS 2016); fixed-view data: TASTE-Rob (CVPR 2025); substantial camera-motion filtering: Generative Image Dynamics (CVPR 2024); fixed-camera test prompts: NewtonGen (ICLR 2026); static-view temporal branch: DimensionX (ICCV 2025). Simple 2D shapes: How Far Is Video Generation from World Model? (ICML 2025); 2D rigid proxies/meshes: PhysGen (ECCV 2024)/PhysAnimator (CVPR 2025); controlled training motions including rolling balls: Force Prompting (NeurIPS 2025).
Reconstruction/inference: Δynamics (CVPR 2026) uses primitives and fixed cameras; IRIS (ECCV 2026) tests balls, cones and pendulums under stable camera placement. AGILE (SIGGRAPH 2026), Consistent4D (ICLR 2024) and V-SysId (L4DC 2022) use static cameras. Physics-as-Inverse-Graphics (ICLR 2020), CoPhy (ICLR 2020) and Visual De-animation (NeurIPS 2017) study controlled systems. Neural Implicit Representations (WACV 2023) assumes static cameras and planar motion; Physically Based Video Editing (Pacific Graphics 2016) uses fixed cameras and static backgrounds. Learning Physics From Video (CVPR 2025) estimates parameters for systems such as pendula, sliding and free fall; VRDP (NeurIPS 2021) models CLEVRER primitives; Galileo (NeurIPS 2015) uses boxes, cylinders and tori.
Compared with single-motion generation or system-specific parameter estimation, we address unified multi-object rigid-body editing across motions, combined interactions, parameter edits and object addition/removal, offering broad applicability and control. Jointly reconstructing source motion, predicting new contacts and cascading effects, and maintaining visual consistency makes the task more demanding.
Extensions could use richer proxies and support surfaces, add moving-camera motion estimation, and expand the motion rules. Stronger model components could substantially improve robustness.
Q1. Latency and interaction.
Table 15 gives approximately 8.24 minutes for reconstruction and 11.47 per edit, including 11.09 for video generation, the main per-edit cost. Reusing the source scene with faster or streaming generators could enable real-time editing.
Q2. Fairness and context.
Our method and general video-editing baselines receive the same source video and quantitative instruction (Sec. 5.1). We infer the scene without extra ground-truth scene parameters or target trajectories. Recovering scene-specific physical information is part of the task and our contribution. We compare complete systems' ability to infer this information from shared inputs and use it for editing; Q1 reports the added cost.
In C.5, baselines receive explicit downstream-motion and interaction descriptions, while VideoPhysEdit retains the original instruction. Results on four tasks show no consistent improvement (Table 16): models still struggle to reliably generate the described physical changes. We believe the main bottleneck is translating physical edits into correct dynamics, beyond prompt detail alone. Our pipeline addresses this by recovering physical information from shared inputs, computing motion through explicit interventions and simulation, and guiding video generation. This motivates our dedicated reconstruction and dynamics-prediction design and is a core contribution.
These results demonstrate our contributions to physical scene reconstruction, intervention and paired evaluation. We respectfully ask you to reassess them in light of these clarifications and results. Thank you for your careful review and discussion.