- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PhysGuide improves the physical plausibility of generated videos through physics-guided preference optimization. It constructs physics prototypes from real videos and textual descriptions, then uses prototype associations to modulate preference rewards and introduce an adaptive margin. Experiments report improvements on PhyGenBench and VideoPhy, alongside improved VBench scores.
Critical strengths
1 The reported improvements are substantial. Compared with standard DPO, PhysGuide improves PhyGenBench from 53.1 to 63.1 and VideoPhy from 40.7 to 45.3. The VBench results also suggest that these gains do not sacrifice general video quality.
2 The ablations provide useful evidence for both reward modulation and the divergence-based margin. Comparisons against random grouping and predefined physics categories further support the proposed prototype construction.
Critical weaknesses
1 The interpretation of prototype distance as physical-violation severity is insufficiently validated. The projection measures association with physical phenomena, which does not necessarily indicate whether those phenomena are depicted correctly. Appendix A.3 demonstrates robustness to color changes, but its comparisons across different videos also change scene content and semantics. Consequently, it does not establish that the distance isolates physical violations or tracks their severity.
2 Important implementation details are missing, including the video encoder and its relationship to the generator, the EMA update procedure, and how the policy log-ratio is computed for the diffusion model. It is also unclear whether gradients flow through the prototype projections and adaptive margin. These details matter because an evolving representation could reduce the margin by changing the representation itself, rather than improving generated physics.
Questions for authors
1 Can you validate the proposed distance using matched scenes with controlled physical violations of increasing severity, while separately varying appearance?
2 Which video encoder and diffusion log-ratio estimator are used? Are gradients stopped through the projections and margin, and how are the encoder and prototypes updated?
3 Were the preference-training baselines evaluated using the same training pairs and comparable optimization budgets?
4 comments
Rebuttal - 2
- Important implementation details are missing
Gradient Flow. We clarify that gradients flow exclusively through the differentiable policy rewards and , updating only the trainable LoRA parameters of the HunyuanVideo model, while the pretrained backbone remains frozen. The physics-principle representations and , and the divergence term , do not receive gradients and are detached from the computational graph.
Consequently, for the scenario , the model cannot directly minimize the preference loss by shrinking the distance between the preferred and rejected representations, which addresses the reviewer's concern about representation collapse. Since the LoRA-adapted video features evolve during training, we update the physics-principle prototypes using an exponential moving average (EMA) of the corresponding cluster embeddings.
Diffusion Likelihood Estimation. We clarify that we do not explicitly compute the exact video likelihoods. Instead, following diffusion/flow-matching DPO [1], we approximate the policy log-likelihood ratio using the difference in velocity-prediction errors between the trainable HunyuanVideo model and a frozen reference model.
For each preferred and rejected video, we encode the video into latent space, sample a random timestep and Gaussian noise, and construct noisy latents using the flow-matching schedule. Both the trainable policy and the reference model predict the velocity from the same noisy latents.
We estimate the implicit policy reward as:
where is the target flow velocity and includes the loss scaling.
This provides a differentiable surrogate for the policy reward, with gradients propagated through the trainable model while the reference model remains frozen. The estimated rewards for the preferred and rejected videos are then used in our principle-guided preference objective (Eq. 8).
EMA Updates. All codes live in the embedding space of an encoder that maps a video to a feature vector in . Instead of keeping this encoder fixed, we update its parameters with an exponential moving average (EMA) of the corresponding encoder parameters in the generator:
where denotes the visual encoder parameters of the generator at DPO step , and is the EMA decay coefficient.
Video and Text Encoder. We clarify that the video encoder and text encoder are part of the pretrained HunyuanVideo model used for preference optimization. The pretrained backbone weights are frozen, while the LoRA parameters are trainable. Video features are extracted from the HunyuanVideo diffusion transformer (DiT) hidden states.
[1]. Wallace, Bram, et al. "Diffusion model alignment using direct preference optimization." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
Thank you for clarifying the detached gradient paths and reward estimator, and for providing the matched PhyGDPO comparison. These address several of my concerns. However, the two-bin PhysicsIQ analysis does not yet resolve my main concern: differences in scene content and physical phenomena may still confound the relationship between divergence and violation severity. Could you provide a controlled within-scene comparison, or narrow the interpretation of this term accordingly? Please also clarify whether EMA is applied to encoder parameters, prototype embeddings, or both. I am currently maintaining my rating pending these clarifications.
Reply
We thank the reviewer for their response and feedback. We discuss the concerns as follows:
Relationship between divergence and violation severity (controlled physical violations of increasing severity) To evaluate the relationship between divergence and violation severity in a controlled within-scene comparison, we select 10 scenes from Physics-IQ (which will be winner videos) and generate 10 loser videos using different HunyuanVideo seeds while keeping the conditioning input fixed. This setting keeps the scene context fixed and only the physics phenomenon changes. For each scene, the generated videos are ranked according to the severity of their physical violations based on PhysicsIQ scores, from 1 (least severe violation, high PhysicsIQ score) to 10 (most severe violation, low PhysicsIQ score). The PhysicsIQ scores are in a range of [0-100] where 100 is better. We then compute the physics-violation divergence (g(xw,xl)) for every winner-loser pair and rank the divergence scores as well. Next, we compute the Spearman score between the ranking and report the results below.
| Scene ID | Physics-IQ Score (Mean ± SD) (High is better) | Divergence g (Mean ± SD) (Low is better) | Spearman rho |
|---|---|---|---|
| S1 | 71.2 ± 10.1 | 0.24 ± 0.12 | -0.867 |
| S2 | 67.5 ± 12.2 | 0.33 ± 0.14 | -0.745 |
| S3 | 60.2 ± 15.8 | 0.35 ± 0.17 | -0.879 |
| S4 | 56.8 ± 17.6 | 0.37 ± 0.16 | -0.733 |
| S5 | 52.1 ± 15.6 | 0.39 ± 0.15 | -0.939 |
| S6 | 43.8 ± 15.9 | 0.46 ± 0.18 | -0.842 |
| S7 | 38.2 ± 18.1 | 0.51 ± 0.21 | -0.988 |
| S8 | 29.4 ± 11.3 | 0.59 ± 0.16 | -0.830 |
| S9 | 25.7 ± 9.5 | 0.64 ± 0.14 | -0.782 |
| S10 | 22.6 ± 8.2 | 0.69 ± 0.12 | -0.939 |
| Mean scene-wise rho | — | — | -0.854 |
From the above table, we can see that the negative within-scene Spearman correlations support that videos with lower physical plausibility scores exhibit greater divergence from their corresponding physically plausible winners.
EMA application. We apply the exponential-moving-average (EMA) to the encoder parameters only and use it to update the physics principle vectors throughout training.
Rebuttal
We thank the reviewer for their thoughtful comments and constructive feedback on the formulation of our physics-principle-guided objective, clarifying the implementation details, and the experimental validation of our approach.
We clarify that our physics-violation divergence term measures differences in association across multiple physics principles, rather than directly measuring the severity of a single physical violation.
Each video is represented by a set of scalar association scores corresponding to different physical principles, such as collision dynamics, momentum conservation, and object interactions. Physically implausible videos may exhibit weaker associations with the relevant principles compared to physically plausible videos. Consequently, videos with more severe physical violations may have smaller association scores than those with milder violations. When comparing physically plausible and implausible videos, larger differences in their principle-association distributions can lead to a higher physics-violation divergence penalty, g(xw,xl).
Empirical validation. To investigate this relationship, we conduct an analysis on the PhysicsIQ test set. We use the PhysicsIQ evaluation error as a proxy for physical violation severity, where higher error values indicate greater deviations from expected physical behavior, while lower values indicate more physically plausible dynamics. For each evaluated video pair, we compute the corresponding physics-violation divergence penalty, g(xw,xl), and examine its relationship with the measured physics error. The results are summarized below:
Our analysis shows that higher physics evaluation errors are associated with larger divergence penalties (0.84 vs. 0.35). This suggests that the proposed divergence term captures principle-level discrepancies that are correlated with the severity of physical violations.