- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PhysPlan is a training-free framework for physically plausible image-to-video generation. A VLM parses the input scene into a graph and plans qualitative object-state changes, which are implemented as typed graph edits and rendered into target keyframes. During video sampling, the edits select manually designed appearance, area, position, depth, and instance-count losses, whose gradients update the video latent while all model weights remain frozen.
Experiments on PhyGenBench and Physics-IQ show improvements over the base model and Frame Guidance. Ablations and a human study provide additional evidence of practical effectiveness.
Critical strengths
-
The matched-keyframe comparison supports the usefulness of selective guidance. PhysPlan and Frame Guidance use the same backbone, keyframes, and anchors, yet PhysPlan improves Physics-IQ from 30.3 to 38.2. This indicates that its benefits extend beyond obtaining better reference images from an external image editor.
-
The system is evaluated from several complementary perspectives. Benchmark scores, component ablations, human preferences, failure attribution, and runtime measurements provide a reasonably comprehensive account of its practical behavior.
Critical weaknesses
-
The physical representation and guidance are heavily hand-designed, limiting their generality. The framework expresses phenomena through discrete objects, a fixed attribute/relation vocabulary, and predefined mappings from edits to visual losses. These constraints naturally capture changes in appearance, size, location, and presence, but provide limited control over continuous dynamics, quantitative physical relationships, or phenomena better represented as fields or distributed materials. The acknowledged weakness in fluid dynamics illustrates this limitation. Moreover, the deterministic checks establish graph consistency rather than physical validity. The evidence therefore supports a useful mechanism for enforcing selected qualitative outcomes more strongly than a general advance in physical reasoning.
-
The methodological advance is limited relative to the complexity of the inference pipeline. Physical planning and target appearance are supplied by pretrained VLM and image-editing models, while the video generator is steered through established latent-optimization machinery and manually constructed visual objectives. The main contribution is consequently the organization and selection of these components, with limited evidence of a broadly applicable new reasoning or optimization principle. This concern is reinforced by evaluation on only one video backbone and substantial inference overhead: approximately 366 seconds and 64 GB per video, versus 100 seconds and 26 GB for the base model. Training-free operation is useful, but does not by itself establish sufficient methodological significance.
Questions for authors
My main concern is an inherent limitation of the formulation: physical guidance is expressed through a hand-designed graph-edit vocabulary and corresponding visual proxy losses, while continuous dynamics between anchors remain unconstrained by explicit physical relationships. What classes of physical phenomena can this representation meaningfully capture, and which fall outside its scope? Please clarify how these representational limitations affect the claimed generality of the method.
5 comments
Response to Reviewer 1Zym (Part 2/2)
W2: Clarifying Methodological Contribution and Cost
Our contribution. PhysPlan is a training-free interface between physical reasoning and a frozen VDM. Its plan is a state graph grounded in the observed frame and changed only by verified typed edits, and this graph decides what each guidance term measures, where the update acts, and when each state should hold. The tools that execute the plan, a VLM, an open-vocabulary segmenter, a depth estimator, a feature encoder, and the latent optimization of Frame Guidance, are standard and widely used across vision; our contribution is the rule that couples them. The other reviewers identify this coupling as the strength of the design: one state graph determines the state changes, loss types, regions, and temporal anchors (Reviewer zzZk), which is a sensible improvement over treating every detail of a generated keyframe as a target (Reviewer xrzv). Prior planners instead derive the plan from the prompt and pass it to the VDM as whole keyframes or trajectories (VLIPP, Chain-of-Events, CausalMotion), and prior guidance matches whole frames (Frame Guidance). Because PhysPlan requires no training data or fine-tuning, it also applies to a new backbone without the per-backbone training that methods such as PhysHPO and PhyGDPO need.
Same tools, smaller gains without the graph. The same tools appear in prior work, but without a grounded plan that decides what each one measures, they yield smaller gains (Table 2). VLIPP uses a VLM planner and Grounded-SAM-2 with a CogVideoX-based generator and reaches 0.60 on PhyGenBench and 34.6 on Physics-IQ. Frame Guidance uses our keyframes and the same latent optimization on the same base, yet stays at 0.51 and 30.3, i.e., it does not improve its base on PhyGenBench (0.52); as the reviewer notes, this matched comparison shows that the benefit of PhysPlan extends beyond better reference images from the image editor. PhysPlan reaches 0.77 and 38.2. Training-based methods improve their T2V base models on PhyGenBench by 0.16 (PhysHPO) and 0.15 (PhyGDPO), whereas PhysPlan improves its I2V base by 0.25 without any training, although T2V and I2V results are not directly comparable.
The components couple rather than stack. The one-factor ablations in Table R6 (response to Reviewer zzZk) show that the gain comes from how the graph connects the components, not from adding them. With identical components, building G_0 from the prompt instead of the observed frame costs 7.1 points, and regenerating the graph instead of editing it costs 7.3 (rows i-a and ii). With the same keyframes and region weighting, replacing the graph-selected terms by whole-frame L2 costs 3.7 points (row iii-a). Removing any single factor costs 1.3 to 7.3 points, and these drops sum to 21.6, about twice the total gain over the base model (11.1), so the factors depend on each other rather than adding independently.
Beyond our original setting. Under the same VLM and generator weights, PhysPlan outperforms VLIPP, and it remains ahead with a different VLM (Table R1, response to Reviewer xrzv). On two more backbones, PhysPlan improves Physics-IQ by a similar margin (Table R4).
Cost. The comparison that isolates our cost is Frame Guidance, which uses the same optimization: PhysPlan requires 18% more runtime than Frame Guidance (Table 14 of the submission), and 9% and 11% more on LTX-Video-I2V and Wan2.2-TI2V-5B (Table R4), mainly for the feature encoder, instance matching, and the additional terms. Peak memory is likewise close to that of Frame Guidance (64 vs. 58 GB on CogVideoX-I2V-5B). Most of the runtime is backpropagation through the denoiser, which every loss-guided method shares.
Kind Reminder
Dear Reviewer 1Zym,
As the discussion period closes on November 18 (AoE), we would like to kindly check whether our response has addressed your concerns: the scope of the representation, now stated explicitly and supported by the per-law analysis (Tables R2–R3), the two additional backbones (Table R4), and the cost relative to Frame Guidance. If so, we would be grateful if you would consider updating your score. We are happy to clarify anything further.
Response to Reviewer 1Zym (Part 1/2)
Thank you for the careful assessment of the scope of PhysPlan, and for noting that the matched comparison with Frame Guidance shows a benefit beyond better reference images from the image editor. Revisions are highlighted in blue in the updated PDF. Your points concern which phenomena the representation can capture (W1, Q1) and the methodological contribution and cost (W2).
W1 & Q1: Clarifying the Scope and Generality of the Representation
Below, we state the scope of PhysPlan precisely, show where it matters, and explain how the representation can be extended.
What PhysPlan targets. PhysPlan enforces the qualitative outcome of a phenomenon in the observed scene: which objects change, into what state, where, and in what order. This is the design stated in the submission, in which the VLM judges only which objects change, how, and in what order, without estimating quantities (Sections 2 and 3.1). PhysPlan therefore does not model continuous dynamics, quantitative relations, or field quantities. In the revised manuscript, the abstract, introduction, and conclusion state this scope, and the contribution is described as enforcing grounded qualitative outcomes in a frozen VDM.
Why this target matters. Many failures that PhyGenBench and Physics-IQ measure are wrong qualitative outcomes, such as an object that vanishes without a product, an object that duplicates, or a flame that keeps burning under water (Figure 4), and these are the outcomes that the graph controls.
Scope (Q1). Table R2 (response to Reviewer xrzv) reports the gain of PhysPlan on each physical law defined by PhyGenBench (Meng et al., 2025). The gains are largest for laws whose outcome is a discrete change of object state, such as a change of color, a phase change, or a collapse (e.g., Atmospheric Pressure +0.49 and Color +0.48), and smallest for laws whose outcome is a light pattern or a medium without a clear object boundary (e.g., Dispersion +0.02 and Tyndall Effect +0.04) or continuous motion (Gravity +0.12). Fields, waves, and quantitative laws are outside the representation.
Continuous phenomena. Part of this limitation lies in how coarsely continuous phenomena are planned rather than in the representation itself. In Table R3 (response to Reviewer xrzv), one added instruction in the stage-1 prompt splits a continuous change into successive intermediate states: the VLM then plans 6.3 instead of 2.6 events per video, and Fluid Dynamics on Physics-IQ improves from 31.8 to 33.9, close to VLIPP (34.1), at 24% more runtime.
Why the vocabulary is fixed. The attribute keys and relation types are fixed, but their values are open-vocabulary (Table 5), so a key such as surface can take values like wet, frosted, or charred. Fixing the keys is what makes an edit checkable and measurable: a key either exists or not, and each key is assigned a loss term. The mapping from edit types to loss terms follows from what each edit changes: appearance, area, location, depth order, or presence. Table 4 of the submission supports this mapping, since each term matters most in the categories it is designed for. Extending the vocabulary, for example with particle sets or fields for liquids, requires a new key and a new measurement, without changing the rest of the pipeline.
Physical validity. As in our response to Reviewer zzZk (W1, P1), the deterministic checks are designed to verify the structure of the reasoning chain rather than physical laws; these structural checks encode basic physical invariants, such as that objects cannot appear or vanish without a source, while the physical content of each change comes from the qualitative rule that the VLM gives. The revised wording makes this distinction explicit.
In summary, PhysPlan targets grounded qualitative outcomes rather than continuous or quantitative dynamics. This scope is now stated explicitly, and it covers the qualitative failures that current VDMs most visibly make.